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IMPROVEMENTS TO AN AGILE NETWORK PROTOCOL 
FOR SECURE COMMUNICATIONS 
WITH ASSURED SYSTEM AVAILABILITY 

5 

CROSS-REFERENCE TO RELATED APPLICATION 

This application claims priority from and is a continuation-in-part of 
previously filed U.S. application serial number 09/429,643, filed on October 29, 1999. 
The subject matter of that application, which is bodily incorporated herein, derives 

10 from provisional U.S. application numbers 60/106,261 (filed October 30, 1998) and 
60/137.704 (filed June 7, 1999). 
BACKGROUND OF THE INVENTION 

A tremendous variety of methods have been proposed and implemented to 
provide security and anonymity for communications over the Internet. The variety 

15 stems, in part, from the different needs of different Internet users. A basic heuristic 
framework to aid in discussing these different security techniques is illustrated in FIG. 
1. Two terminals, an originating terminal 100 and a destination terminal 110 are in 
communication over the Internet. It is desired for the communications to be secure, 
that is, immune to eavesdropping. For example, terminal 100 may transmit secret 

20 information to terminal 110 over the Internet 107. Also, it may be desired to prevent 
an eavesdropper from discovering that terminal 100 is in communication with 
terminal 110. For example, if terminal 100 is a user and terminal 1 10 hosts a web site, 
terminal 100\s user may not want anyone in the intervening networks to know what 
web sites he is "visiting." Anonymity would thus be an issue, for example, for 

25 companies that want to keep their market research interests private and thus would 
prefer to prevent outsiders from knowing which web-sites or other Internet resources 
they are "visiting." These two security issues may be called data security and 
anonymity, respectively. 

Data security is usually tackled using some form of data encryption. An 

30 encryption key 48 is known at both the originating and terminating terminals 100 and 
1 10. The keys may be private and public at the originating and destination terminals 
100 and 110. respectively or they may be symmetrical keys (the same key is used by 
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both parties to encrypt and decrypt). Many encryption methods are known and usable 
in this context. 

To hide traffic from a local administrator or ISP, a user can employ a local 
proxy server in communicating over an encrypted channel with an outside proxy such 
5 that the local administrator or ISP only sees the encrypted traffic. Proxy servers 
prevent destination servers from determining the identities of the originating clients. 
This system employs an intermediate server interposed between client and destination 
server. The destination server sees only the Internet Protocol (IP) address of the proxy 
server and not the originating client. The target server only sees the address of the 

10 outside proxy. This scheme relies on a trusted outside proxy server. Also, proxy 
schemes are vulnerable to traffic analysis methods of determining identities of 
transmitters and receivers. Another important limitation of proxy servers is that the 
server knows the identities of both calling and called parties. In many instances, an 
originating terminal, such as terminal A, would prefer to keep its identity concealed 

15 from the proxy, for example, if the proxy server is provided by an Internet service 
provider (ISP). 

To defeat traffic analysis, a scheme called Chaunvs mixes employs a proxy 
server that transmits and receives fixed length messages, including dummy messages. 
Multiple originating terminals are connected through a mix (a server) to multiple 

20 target servers. It is difficult to tell which of the originating terminals are 
. communicating to which of the connected target servers, and the dummy messages 
confuse eavesdroppers' efforts to detect communicating pairs by analyzing traffic. A 
drawback is that there is a risk that the mix server could be compromised. One way to 
deal with this risk is to spread the trust among multiple mixes. If one mix is 

25 compromised, the identities of the originating and target terminals may remain 
concealed. This strategy requires a number of alternative mixes so that the 
intermediate servers interposed between the originating and target terminals are not 
determinable except by compromising more than one mix. The strategy wraps the 
message with multiple layers of encrypted addresses. The first mix in a sequence can 

30 decrypt only the outer layer of the message to reveal the next destination mix in 
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sequence. The second mix can decrypt the message to reveal the next mix and so on. 
The target server receives the message and, optionally, a multi-layer encrypted 
payload containing return information to send data back in the same fashion. The only 
way to defeat such a mix scheme is to collude among mixes. If the packets are all 
5 fixed-length and intermixed with dummy packets, there is no way to do any kind of 
traffic analysis. 

Still another anonymity technique, called 'crowds/ protects the identity of the 
originating terminal from the intermediate proxies by providing that originating 
terminals belong to groups of proxies called crowds. The crowd proxies are 

10 interposed between originating and target terminals. Each proxy through which the 
message is sent is randomly chosen by an upstream proxy. Each intermediate proxy 
can send the message either to another randomly chosen proxy in the fcfc crowd" or to 
the destination. Thus, even crowd members cannot determine if a preceding proxy is 
the originator of the message or if it was simply passed from another proxy. 

15 ZKS (Zero-Knowledge Systems) Anonymous IP Protocol allows users to 

select up to any of five different pseudonyms, while desktop software encrypts 
outgoing traffic and wraps it in User Datagram Protocol (UDP) packets. The first 
server in a 2+-hop system gets the UDP packets, strips off one layer of encryption to 
add another, then sends the traffic to the next server, which strips off yet another layer 

20 of encryption and adds a new one. The user is permitted to control the number of 
hops. At the final server, traffic is decrypted with an untraceable IP address. The 
technique is called onion-routing. This method can be defeated using traffic analysis. 
For a simple example, bursts of packets from a user during low-duty periods can 
reveal the identities of sender and receiver. 

25 Firewalls attempt to protect LANs from unauthorized access and hostile 

exploitation or damage to computers connected to the LAN. Firewalls provide a 
server through which all access to the LAN must pass. Firewalls are centralized 
systems that require administrative overhead to maintain. They can be compromised 
by virtual-machine applications (''applets''). They instill a false sense of security that 

30 leads to security breaches for example by users sending sensitive information to 
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servers outside the firewall or encouraging use of modems to sidestep the firewall 
security. Firewalls are not useful for distributed systems such as business travelers, 
extranets, small teams, etc. 
SUMMARY OF THE INVENTION 
5 A secure mechanism for communicating over the internet, including a protocol 

referred to as the Tunneled Agile Routing Protocol (TARP), uses a unique two-layer 
encryption format and special TARP routers. TARP routers are similar in function to 
regular IP routers. Each TARP router has one or more IP addresses and uses normal 
IP protocol to send IP packet messages ("packets" or "datagrams"). The IP packets 

1 0 exchanged between TARP terminals via TARP routers are actually encrypted packets 
whose true destination address is concealed except to TARP routers and servers. The 
normal or %k clear" or "outside" IP header attached to TARP IP packets contains only 
the address of a next hop router or destination server. That is, instead of indicating a 
final destination in the destination field of the IP header, the TARP packet's IP header 

15 always points to a next-hop in a series of TARP router hops, or to the final 
destination. This means there is no overt indication from an intercepted TARP packet 
of the true destination of the TARP packet since the destination could always be next- 
hop TARP router as well as the final destination. 

Each TARP packet's true destination is concealed behind a layer of encryption 

20 generated using a link key. The link key is the encryption key used for encrypted 
communication between the hops intervening between an originating TARP terminal 
and a destination TARP terminal. Each TARP router can remove the outer layer of 
encryption to reveal the destination router for each TARP packet. To identify the link 
key needed to decrypt the outer layer of encryption of a TARP packet, a receiving 

25 TARP or routing terminal may identify the transmitting terminal by the 
sender/receiver IP numbers in the cleartext IP header. 

Once the outer layer of encryption is removed, the TARP router determines 
the final destination. Each TARP packet 140 undergoes a minimum number of hops to 
help foil traffic analysis. The hops may be chosen at random or by a fixed value. As a 

30 result, each TARP packet may make random trips among a number of geographically 



4 



WO 01/61922 



disparate routers before reaching its destination. Each trip is highly likely to be 
different for each packet composing a given message because each trip is 
independently randomly determined. This feature is called agile routing. The fact that 
different packets take different routes provides distinct advantages by making it 
5 difficult for an interloper to obtain all the packets forming an entire multi-packet 
message. The associated advantages have to do with the inner layer of encryption 
discussed below. Agile routing is combined with another feature that furthers this 
purpose; a feature that ensures that any message is broken into multiple packets. 

The IP address of a TARP router can be changed, a feature called IP agility. 

10 Each TARP router, independently or under direction from another TARP terminal or 
router, can change its IP address. A separate, unchangeable identifier or address is 
also defined. This address, called the TARP address, is known only to TARP routers 
and terminals and may be correlated at any time by a TARP router or a TARP 
terminal using a Lookup Table (LUT). When a TARP router or terminal changes its 

15 IP address, it updates the other TARP routers and terminals which in turn update their 
respective LUTs. 

The message payload is hidden behind an inner layer of encryption in the 
TARP packet that can only be unlocked using a session key. The session key is not 
available to any of the intervening TARP routers. The session key is used to decrypt 
20 the payloads of the TARP packets permitting the data stream to be reconstructed. 

Communication may be made private using link and session keys, which in 
turn may be shared and used according to any desired method. For example, 
public/private keys or symmetric keys may be used. 

To transmit a data stream, a TARP originating terminal constructs a series of 
25 TARP packets from a series of IP packets generated by a network (IP) layer process. 
("Note that the terms "network layer/* "data link layer." "application layer/* etc. used 
in this specification correspond to the Open Systems Interconnection (OSI) network 
terminology.) The payloads of these packets are assembled into a block and chain- 
block encrypted using the session key. This assumes, of course, that all the IP packets 
30 1 are destined for the same TARP terminal. The block is then interleaved and the 
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interleaved encrypted block is broken into a series of payloads, one for each TARP 
packet to be generated. Special TARP headers IP T are then added to each payload 
using the IP headers from the data stream packets. The TARP headers can be identical 
to normal IP headers or customized in some way. They should contain a formula or 

5 data for deinterleaving the data at the destination TARP terminal, a time-to-live (TTL) 
parameter to indicate the number of hops still to be executed, a data type identifier 
which indicates whether the payload contains, for example, TCP or UDP data, the 
sender's TARP address, the destination TARP address, and an indicator as to whether 
the packet contains real or decoy data or a formula for filtering out decoy data if 

1 0 decoy data is spread in some way through the TARP payload data. 

Note that although chain-block encryption is discussed here with reference to 
the session key, any encryption method may be used. Preferably, as in chain block 
encryption, a method should be used that makes unauthorized decryption difficult 
without an entire result of the encryption process. Thus, by separating the encrypted 

1 5 block among multiple packets and making it difficult for an interloper to obtain access 
to all of such packets, the contents of the communications are provided an extra layer 
of security. 

Decoy or dummy data can be added to a stream to help foil traffic analysis by 
reducing the peak-to-average network load. It may be desirable to provide the TARP 

20 process with an ability to respond to the time of day or other criteria to generate more 
decoy data during low traffic periods so that communication bursts at one point in the 
Internet cannot be tied to communication bursts at another point to reveal the 
communicating endpoints. 

Dummy data also helps to break the data into a larger number of 

25 inconspicuously-sized packets permitting the interleave window size to be increased 
while maintaining a reasonable size for each packet. (The packet size can be a single 
standard size or selected from a fixed range of sizes.) One primary reason for desiring 
for each message to be broken into multiple packets is apparent if a chain block 
encryption scheme is used to form the first encryption layer prior to interleaving. A 

30 single block encryption may be applied to portion, or entirety, of a message, and that 
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portion or entirety then interleaved into a number of separate packets. Considering the 
agile IP routing of the packets, and the attendant difficulty of reconstructing an entire 
sequence of packets to form a single block-encrypted message element, decoy packets 
can significantly increase the difficulty of reconstructing an entire data stream. 
5 The above scheme may be implemented entirely by processes operating 

between the data link layer and the network layer of each server or terminal 
participating in the TARP system. Because the encryption system described above is 
insertable between the data link and network layers, the processes involved in 
supporting the encrypted communication may be completely transparent to processes 

10 at the IP (network) layer and above. The TARP processes may also be completely 
transparent to the data link layer processes as well. Thus, no operations at or above 
the Network layer, or at or below the data link layer, are affected by the insertion of 
the TARP stack. This provides additional security to all processes at or above the 
network layer, since the difficulty of unauthorized penetration of the network layer 

15 (by. for example, a hacker) is increased substantially. Even newly developed servers 
running at the session layer leave all processes below the session layer vulnerable to 
attack. Note that in this architecture, security is distributed. That is, notebook 
computers used by executives on the road, for example, can communicate over the 
Internet without any compromise in security. 

20 IP address changes made by TARP terminals and routers can be done at 

regular intervals, at random intervals, or upon detection of "attacks.'" The variation of 
IP addresses hinders traffic analysis that might reveal which computers are 
communicating, and also provides a degree of immunity from attack. The level of 
immunity from attack is roughly proportional to the rate at which the IP address of the 

25 host is changing. 

As mentioned. IP addresses may be changed in response to attacks. An attack 
may be revealed, for example, by a regular series of messages indicating that a router 
is being probed in some way. Upon detection of an attack, the TARP layer process 
may respond to this event by changing its IP address. In addition, it may create a 
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subprocess that maintains the original IP address and continues interacting with the 
attacker in some manner. 

Decoy packets may be generated by each TARP terminal on some basis 
determined by an algorithm. For example, the algorithm may be a random one which 
5 calls for the generation of a packet on a random basis when the terminal is idle. 
Alternatively, the algorithm may be responsive to time of day or detection of low 
traffic to generate more decoy packets during low traffic times. Note that packets are 
preferably generated in groups, rather than one by one, the groups being sized to 
simulate real messages. In addition, so that decoy packets may be inserted in normal 

10 TARP message streams, the background loop may have a latch that makes it more 
likely to insert decoy packets when a message stream is being received. Alternatively, 
if a large number of decoy packets is received along with regular TARP packets, the 
algorithm may increase the rate of dropping of decoy packets rather than forwarding 
them. The result of dropping and generating decoy packets in this way is to make the 

15 apparent incoming message size different from the apparent outgoing message size to 
help foil traffic analysis. 

In various other embodiments of the invention, a scalable version of the 
system may be constructed in which a plurality of IP addresses are preassigned to 
each pair of communicating nodes in the network. Each pair of nodes agrees upon an 

20 algorithm for "hopping" between IP addresses (both sending and receiving), such that 
an eavesdropper sees apparently continuously random IP address pairs (source and 
destination) for packets transmitted between the pair. Overlapping or "reusable" IP 
addresses may be allocated to different users on the same subnet, since each node 
merely verifies that a particular packet includes a valid source/destination pair from 

25 the agreed-upon algorithm. Source/destination pairs are preferably not reused 
between any two nodes during any given end-to-end session, though limited IP block 
sizes or lengthy sessions might require it. 

Further improvements described in this continuation-in-part application 
include: (1) a load balancer that distributes packets across different transmission paths 

30 according to transmission path quality; (2) a DNS proxy server that transparently 
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creates a virtual private network in response to a domain name inquiry; (3) a large-to- 
small link bandwidth management feature that prevents denial-of-service attacks at 
system chokepoints; (4) a traffic limiter that regulates incoming packets by limiting 
the rate at which a transmitter can be synchronized with a receiver; and (5) a signaling 
5 synchronizer that allows a large number of nodes to communicate with a central node 
by partitioning the communication function between two separate entities 
BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is an illustration of secure communications over the Internet according 
to a prior art embodiment. 
10 FIG. 2 is an illustration of secure communications over the Internet according 

to a an embodiment of the invention. 

FIG. 3a is an illustration of a process of forming a tunneled IP packet 
according to an embodiment of the invention. 

FIG. 3b is an illustration of a process of forming a tunneled IP packet 
15 according to another embodiment of the invention. 

FIG. 4 is an illustration of an OSI layer location of processes that may be used 
to implement the invention. 

FIG. 5 is a flow chart illustrating a process for routing a tunneled packet 
according to an embodiment of the invention. 
20 FIG. 6 is a flow chart illustrating a process for forming a tunneled packet 

according to an embodiment of the invention. 

FIG. 7 is a flow chart illustrating a process for receiving a tunneled packet 
according to an embodiment of the invention. 

FIG. 8 shows how a secure session is established and synchronized between a 
25 client and a TARP router. 

FIG. 9 shows an IP address hopping scheme between a client computer and 
TARP router using transmit and receive tables in each computer. 

FIG. 10 shows physical link redundancy among three Internet Service 
Providers (ISPs) and a client computer. 
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FIG. 1 1 shows how multiple IP packets can be embedded into a single "frame' 5 
such as an Ethernet frame, and further shows the use of a discriminator field to 
camouflage true packet recipients. 

FIG. 12A shows a system that employs hopped hardware addresses, hopped IP 
5 addresses, and hopped discriminator fields. 

FIG. 12B shows several different approaches for hopping hardware addresses, 
IP addresses, and discriminator fields in combination. 

FIG. 13 shows a technique for automatically re-establishing synchronization 
between sender and receiver through the use of a partially public sync value. 
10 FIG. 14 shows a '"checkpoint" scheme for regaining synchronization between 

a sender and recipient. 

FIG. 15 shows further details of the checkpoint scheme of FIG. 14. 

FIG. 16 shows how two addresses can be decomposed into a plurality of 
segments for comparison with presence vectors. 
1 5 FIG. 1 7 shows a storage array for a receiver's active addresses. 

FIG. 18 shows the receiver's storage array after receiving a sync request. 

FIG. 19 shows the receiver's storage array after new addresses have been 
generated. 

FIG. 20 shows a system employing distributed transmission paths. 
20 FIG. 21 shows a plurality of link transmission tables that can be used to route 

packets in the system of FIG. 20. 

FIG. 22A shows a flowchart for adjusting weight value distributions 
associated with a plurality of transmission links. 

FIG. 22B shows a flowchart for setting a weight value to zero if a transmitter 
25 turns off. 

FIG. 23 shows a system employing distributed transmission paths with 
adjusted weight value distributions for each path. 

FIG. 24 shows an example using the system of FIG. 23. 
FIG. 25 shows a conventional domain-name look-up service. 
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FIG. 26 shows a system employing a DNS proxy server with transparent VPN 
creation. 

FIG. 27 shows steps that can be carried out to implement transparent VPN 
creation based on a DNS look-up function. 
5 FIG. 28 shows a system including a link guard function that prevents packet 

overloading on a low-bandwidth link LOW BW. 

FIG. 29 shows one embodiment of a system employing the principles of FIG. 

28. 

FIG. 30 shows a system that regulates packet transmission rates by throttling 

1 0 the rate at which synchronizations are performed. 

FIG. 31 shows a signaling server 3101 and a transport server 3102 used to 
establish a VPN with a client computer. 

FIG. 32 shows message flows relating to synchronization protocols of FIG. 31 . 
DETAILED DESCRIPTION OF THE INVENTION 

15 Referring to FIG. 2, a secure mechanism for communicating over the internet 

employs a number of special routers or servers, called TARP routers 122-127 that are 
similar to regular IP routers 128-132 in that each has one or more IP addresses and 
uses normal IP protocol to send normal-looking IP packet messages, called TARP 
packets 140. TARP packets 140 are identical to normal IP packet messages that are 

20 routed by regular IP routers 128-132 because each TARP packet 140 contains a 
destination address as in a normal IP packet. However, instead of indicating a final 
destination in the destination field of the IP header, the TARP packet's 140 IP header 
always points to a next-hop in a series of TARP router hops, or the final destination, 
TARP terminal 1 10. Because the header of the TARP packet contains only the next- 

25 hop destination, there is no overt indication from an intercepted TARP packet of the 
true destination of the TARP packet 140 since the destination could always be the 
next-hop TARP router as well as the final destination, TARP terminal 1 10. 

Each TARP packet's true destination is concealed behind an outer layer of 
encryption generated using a link key 146. The link key 146 is the encryption key 

30 used for encrypted communication between the end points (TARP terminals or TARP 
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routers) of a single link in the chain of hops connecting the originating TARP terminal 
100 and the destination TARP terminal 110. Each TARP router 122-127, using the 
link key 146 it uses to communicate with the previous hop in a chain, can use the link 
key to reveal the true destination of a TARP packet. To identify the link key needed to 
5 decrypt the outer layer of encryption of a TARP packet, a receiving TARP or routing 
terminal may identify the transmitting terminal (which may indicate the link key used) 
by the sender field of the clear IP header. Alternatively, this identity may be hidden 
behind another layer of encryption in available bits in the clear IP header. Each TARP 
router, upon receiving a TARP message, determines if the message is a TARP 

10 message by using authentication data in the TARP packet. This could be recorded in 
available bytes in the TARP packet's IP header. Alternatively, TARP packets could 
be authenticated by attempting to decrypt using the link key 146 and determining if 
the results are as expected. The former may have computational advantages because it 
does not involve a decryption process. 

15 Once the outer layer of decryption is completed by a TARP router 122-127, 

the TARP router determines the final destination. The system is preferably designed 
to cause each TARP packet 140 to undergo a minimum number of hops to help foil 
traffic analysis. The time to live counter in the IP header of the TARP message may 
be used to indicate a number of TARP router hops yet to be completed. Each TARP 

20 router then would decrement the counter and determine from that whether it should 
forward the TARP packet 140 to another TARP router 122-127 or to the destination 
TARP terminal 110. If the time to live counter is zero or below zero after 
decrementing, for an example of usage, the TARP router receiving the TARP packet 
140 may forward the TARP packet 140 to the destination TARP terminal 110. If the 

25 time to live counter is above zero after decrementing, for an example of usage, the 
TARP router receiving the TARP packet 140 may forward the TARP packet 140 to a 
TARP router 122-127 that the current TARP terminal chooses at random. As a result, 
each TARP packet 1 40 is routed through some minimum number of hops of TARP 
routers 122-127 which are chosen at random. 
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Thus, each TARP packet, irrespective of the traditional factors determining 
traffic in the Internet, makes random trips among a number of geographically 
disparate routers before reaching its destination and each trip is highly likely to be 
different for each packet composing a given message because each trip is 
5 independently randomly determined as described above. This feature is called agile 
routing. For reasons that will become clear shortly, the fact that different packets take 
different routes provides distinct advantages by making it difficult for an interloper to 
obtain all the packets forming an entire multi-packet message. Agile routing is 
combined with another feature that furthers this purpose, a feature that ensures that 
10 any message is broken into multiple packets. 

A TARP router receives a TARP packet when an IP address used by the 
TARP router coincides with the IP address in the TARP packet's IP header IP C . The 
IP address of a TARP router, however, may not remain constant. To avoid and 
manage attacks, each TARP router, independently or under direction from another 
15 TARP terminal or router, may change its IP address. A separate, unchangeable 
identifier or address is also defined. This address, called the TARP address, is known 
only to TARP routers and terminals and may be correlated at any time by a TARP 
router or a TARP terminal using a Lookup Table (LUT). When a TARP router or 
terminal changes its IP address, it updates the other TARP routers and terminals 
20 which in turn update their respective LUTs. In reality, whenever a TARP router looks 
up the address of a destination in the encrypted header, it must convert a TARP 
address to a real IP address using its LUT. 

While every TARP router receiving a TARP packet has the ability to 
determine the packet's final destination, the message payload is embedded behind an 
25 inner layer of encryption in the TARP packet that can only be unlocked using a 
session key. The session key is not available to any of the TARP routers 122-127 
intervening between the originating 100 and destination 110 TARP terminals. The 
session key is used to decrypt the pay loads of the TARP packets 140 permitting an 
entire message to be reconstructed. 
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In one embodiment, communication may be made private using link and 
session keys, which in turn may be shared and used according any desired method. 
For example, a public key or symmetric keys may be communicated between link or 
session endpoints using a public key method. Any of a variety of other mechanisms 
5 for securing data to ensure that only authorized computers can have access to the 
private information in the TARP packets 140 may be used as desired. 

Referring to FIG. 3a, to construct a series of TARP packets, a data stream 300 
of IP packets 207a, 207b, 207c, etc., such series of packets being formed by a network 
(IP) layer process, is broken into a series of small sized segments. In the present 
10 example, equal-sized segments 1-9 are defined and used to construct a set of 
interleaved data packets A, B, and C. Here it is assumed that the number of 
interleaved packets A, B, and C formed is three and that the number of IP packets 
207a-207c used to form the three interleaved packets A, B, and C is exactly three. Of 
course, the number of IP packets spread over a group of interleaved packets may be 
15 any convenient number as may be the number of interleaved packets over which the 
incoming data stream is spread. The latter, the number of interleaved packets over 
which the data stream is spread, is called the interleave window. 

To create a packet, the transmitting software interleaves the normal IP packets 
207a el. seq. to form a new set of interleaved payload data 320. This payload data 320 
20 is then encrypted using a session key to form a set of session-key-encrypted payload 
data 330, each of which, A, B, and C, will form the payload of a TARP packet. Using 
the IP header data, from the original packets 207a-207c, new TARP headers IP T are 
formed. The TARP headers IP T can be identical to normal IP headers or customized in 
some way. In a preferred embodiment, the TARP headers IP T are IP headers with 
25 added data providing the following information required for routing and 
reconstruction of messages, some of which data is ordinarily, or capable of being, 
contained in normal IP headers: 

1 . A window sequence number - an identifier that indicates where the 

packet belongs in the original message sequence. 
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2. An interleave sequence number - an identifier that indicates the 
interleaving sequence used to form the packet so that the packet can be 
deinterleaved along with other packets in the interleave window. 

3. A time-to-live (TTL) datum - indicates the number of TARP- 
5 router-hops to be executed before the packet reaches its destination. Note 

that the TTL parameter may provide a datum to be used in a probabilistic 
formula for determining whether to route the packet to the destination or to 
another hop. 

4. Data type identifier - indicates whether the payload contains, for 
1 0 example, TCP or UDP data. 

5. Sender's address - indicates the sender's address in the TARP 
network. 

6. Destination address - indicates the destination terminal's address in 
the TARP network. 

15 7. Decoy/Real - an indicator of whether the packet contains real 

message data or dummy decoy data or a combination. 
Obviously, the packets going into a single interleave window must include 
only packets with a common destination. Thus, it is assumed in the depicted example 
that the IP headers of IP packets 207a-207c all contain the same destination address or 

20 at least will be received by the same terminal so that they can be deinterleaved. Note 
that dummy or decoy data or packets can be added to form a larger interleave window 
than would otherwise be required by the size of a given message. Decoy or dummy 
data can be added to a stream to help foil traffic analysis by leveling the load on the 
network. Thus, it may be desirable to provide the TARP process with an ability to 

25 respond to the time of day or other criteria to generate more decoy data during low 
traffic periods so that communication bursts at one point in the Internet cannot be tied 
to communication bursts at another point to reveal the communicating endpoints. 

Dummy data also helps to break the data into a larger number of 
inconspicuously-sized packets permitting the interleave window size to be increased 

30 while maintaining a reasonable size for each packet, (The packet size can be a single 
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standard size or selected from a fixed range of sizes.) One primary reason for desiring 
for each message to bet broken into multiple packets is apparent if a chain block 
encryption scheme is used to form the first encryption layer prior to interleaving. A 
single block encryption may be applied to a portion, or the entirety, of a message, and 
5 that portion or entirety then interleaved into a number of separate packets. 

Referring to FIG. 3b, in an alternative mode of TARP packet construction, a 
series of IP packets are accumulated to make up a predefined interleave window. The 
payloads of the packets are used to construct a single block 520 for chain block 
encryption using the session key. The payloads used to form the block are presumed 

10 to be destined for the same terminal. The block size may coincide with the interleave 
window as depicted in the example embodiment of FIG. 3b. After encryption, the 
encrypted block is broken into separate payloads and segments which are interleaved 
as in the embodiment of Fig 3a. The resulting interleaved packets A, B, and C, are 
then packaged as TARP packets with TARP headers as in the Example of FIG. 3a. 

15 The remaining process is as shown in, and discussed with reference to, FIG. 3a. 

Once the TARP packets 340 are formed, each entire TARP packet 340, 
including the TARP header IP T , is encrypted using the link key for communication 
with the first-hop-TARP router. The first hop TARP router is randomly chosen. A 
final unencrypted IP header IP C is added to each encrypted TARP packet 340 to form 

20 a normal IP packet 360 that can be transmitted to a TARP router. Note that the 
process of constructing the TARP packet 360 does not have to be done in stages as 
described. The above description is just a useful heuristic for describing the final 
product, namely, the TARP packet. 

Note that, TARP header IP T could be a completely custom header 

25 configuration with no similarity to a normal IP header except that it contain the 
information identified above. This is so since this header is interpreted by only TARP 
routers. 

The above scheme may be implemented entirely by processes operating 
between the data link layer and the network layer of each server or terminal 
30 participating in the TARP system. Referring to FIG. 4, a TARP transceiver 405 can be 
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an originating terminal 100, a destination terminal 1 10, or a TARP router 122-127. In 
each TARP Transceiver 405, a transmitting process is generated to receive normal 
packets from the Network (IP) layer and generate TARP packets for communication 
over the network. A receiving process is generated to receive normal IP packets 
5 containing TARP packets and generate from these normal IP packets which are 
"passed up" to the Network (IP) layer. Note that where the TARP Transceiver 405 is a 
router, the received TARP packets 140 are not processed into a stream of IP packets 
415 because they need only be authenticated as proper TARP packets and then passed 
to another TARP router or a TARP destination terminal 1 10. The intervening process, 
10 a "TARP Layer" 420, could be combined with either the data link layer 430 or the 
Network layer 410. In either case, it would intervene between the data link layer 430 
so that the process would receive regular IP packets containing embedded TARP 
packets and "hand up" a series of reassembled IP packets to the Network layer 410. 
As an example of combining the TARP layer 420 with the data link layer 430, a 
1 5 program may augment the normal processes running a communications card, for 
example, an Ethernet card. Alternatively, the TARP layer processes may form part of 
a dynamically loadable module that is loaded and executed to support 
communications between the network and data link layers. 

Because the encryption system described above can be inserted between the 
20 data link and network layers, the processes involved in supporting the encrypted 
communication may be completely transparent to processes at the IP (network) layer 
and above. The TARP processes may also be completely transparent to the data link 
layer processes as well. Thus, no operations at or above the network layer, or at or 
below the data link layer, are affected by the insertion of the TARP stack. This 
25 provides additional security to all processes at or above the network layer, since the 
difficulty of unauthorized penetration of the network layer (by. for example, a hacker) 
is increased substantially. Even newly developed servers running at the session layer 
leave all processes below the session layer vulnerable to attack. Note that in this 
architecture, security is distributed. That is, notebook computers used by executives 
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on the road, for example, can communicate over the Internet without any compromise 
in security. 

Note that IP address changes made by TARP terminals and routers can be 
done at regular intervals, at random intervals, or upon detection of "attacks." The 
variation of IP addresses hinders traffic analysis that might reveal which computers 
are communicating, and also provides a degree of immunity from attack. The level of 
immunity from attack is roughly proportional to the rate at which the IP address of the 
host is changing. 

As mentioned, IP addresses may be changed in response to attacks. An attack 
may be revealed, for example, by a regular series of messages indicates that a router is 
being probed in some way. Upon detection of an attack, the TARP layer process may 
respond to this event by changing its IP address. To accomplish this, the TARP 
process will construct a TARP-formatted message, in the style of Internet Control 
Message Protocol (ICMP) datagrams as an example; this message will contain the 
machine's TARP address, its previous IP address, and its new IP address. The TARP 
layer will transmit this packet to at least one known TARP router; then upon receipt 
and validation of the message, the TARP router will update its LUT with the new IP 
address for the stated TARP address. The TARP router will then format a similar 
message, and broadcast it to the other TARP routers so that they may update their 
LUTs. Since the total number of TARP routers on any given subnet is expected to be 
relatively small, this process of updating the LUTs should be relatively fast. It may 
not, however, work as well when there is a relatively large number of TARP routers 
and/or a relatively large number of clients; this has motivated a refinement of this 
architecture to provide scalability; this refinement has led to a second embodiment, 
which is discussed below. 

Upon detection of an attack, the TARP process may also create a subprocess 
that maintains the original IP address and continues interacting with the attacker. The 
latter may provide an opportunity to trace the attacker or study the attacker's methods 
(called "fishbowling" drawing upon the analogy of a small fish in a fish bowl that 
"thinks" it is in the ocean but is actually under captive observation). A history of the 
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communication between the attacker and the abandoned (fishbowled) IP address can 
be recorded or transmitted for human analysis or further synthesized for purposes of 
responding in some way. 

As mentioned above, decoy or dummy data or packets can be added to 
outgoing data streams by TARP terminals or routers. In addition to making it 
convenient to spread data over a larger number of separate packets, such decoy 
packets can also help to level the load on inactive portions of the Internet to help foil 
traffic analysis efforts. 

Decoy packets may be generated by each TARP terminal 100, 110 or each 
router 122-127 on some basis determined by an algorithm. For example, the algorithm 
may be a random one which calls for the generation of a packet on a random basis 
when the terminal is idle. Alternatively, the algorithm may be responsive to time of 
day or detection of low traffic to generate more decoy packets during low traffic 
times. Note that packets are preferably generated in groups, rather than one by one, 
the groups being sized to simulate real messages. In addition, so that decoy packets 
may be inserted in normal TARP message streams, the background loop may have a 
latch that makes it more likely to insert decoy packets when a message stream is being 
received. That is, when a series of messages are received, the decoy packet generation 
rate may be increased. Alternatively, if a large number of decoy packets is received 
along with regular TARP packets, the algorithm may increase the rate of dropping of 
decoy packets rather than forwarding them. The result of dropping and generating 
decoy packets in this way is to make the apparent incoming message size different 
from the apparent outgoing message size to help foil traffic analysis. The rate of 
reception of packets, decoy or otherwise, may be indicated to the decoy packet 
dropping and generating processes through perishable decoy and regular packet 
counters. (A perishable counter is one that resets or decrements its value in response 
to time so that it contains a high value when it is incremented in rapid succession and 
a small value when incremented either slowly or a small number of times in rapid 
succession.) Note that destination TARP terminal 110 may generate decoy packets 
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equal in number and size to those TARP packets received to make it appear it is 
merely routing packets and is therefore not the destination terminal. 

Referring to FIG. 5, the following particular steps may be employed in the 
above-described method for routing TARP packets. 

• SO. A background loop operation is performed which applies an algorithm which 
determines the generation of decoy IP packets. The loop is interrupted when an 
encrypted TARP packet is received. 

• S2. The TARP packet may be probed in some way to authenticate the packet 
before attempting to decrypt it using the link key. That is, the router may 
determine that the packet is an authentic TARP packet by performing a selected 
operation on some data included with the clear IP header attached to the encrypted 
TARP packet contained in the payload. This makes it possible to avoid 
performing decryption on packets that are not authentic TARP packets. 

• S3. The TARP packet is decrypted to expose the destination TARP address and an 
indication of whether the packet is a decoy packet or part of a real message. 

• S4. If the packet is a decoy packet, the perishable decoy counter is incremented. 

• S5. Based on the decoy generation/dropping algorithm and the perishable decoy 
counter value, if the packet is a decoy packet, the router may choose to throw it 
away. If the received packet is a decoy packet and it is determined that it should 
be thrown away (S6), control returns to step SO. 

• S7. The TTL parameter of the TARP header is decremented and it is determined if 
the TTL parameter is greater than zero. 

• S8. If the TTL parameter is greater than zero, a TARP address is randomly chosen 
from a list of TARP addresses maintained by the router and the link key and IP 
address corresponding to that TARP address memorized for use in creating a new 
IP packet containing the TARP packet. 

• S9. If the TTL parameter is zero or less, the link key and IP address corresponding 
to the TARP address of the destination are memorized for use in creating the new 
IP packet containing the TARP packet. 
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• S10. The TARP packet is encrypted using the memorized link key. 

• SI 1. An IP header is added to the packet that contains the stored IP address, the 
encrypted TARP packet wrapped with an IP header, and the completed packet 
transmitted to the next hop or destination. 

5 

Referring to FIG. 6, the following particular steps may be employed in the 
above-described method for generating TARP packets. 

• S20. A background loop operation applies an algorithm that determines the 
10 generation of decoy IP packets. The loop is interrupted when a data stream 

containing IP packets is received for transmission. 

• S21 . The received IP packets are grouped into a set consisting of messages with a 
constant IP destination address. The set is further broken down to coincide with a 
maximum size of an interleave window The set is encrypted, and interleaved into 

15 a set of payloads destined to become TARP packets. 

• S22. The TARP address corresponding to the IP address is determined from a 
lookup table and stored to generate the TARP header. An initial TTL count is 
generated and stored in the header. The TTL count may be random with minimum 
and maximum values or it may be fixed or determined by some other parameter. 

20 • S23. The window sequence numbers and interleave sequence numbers are 
recorded in the TARP headers of each packet. 

• S24. One TARP router address is randomly chosen for each TARP packet and the 
IP address corresponding to it stored for use in the clear IP header. The link key 
corresponding to this router is identified and used to encrypt TARP packets 

25 containing interleaved and encrypted data and TARP headers. 

• S25. A clear IP header with the first hop router's real IP address is generated and 
added to each of the encrypted TARP packets and the resulting packets. 

Referring to FIG. 7, the following particular steps may be employed in the 
30 above-described method for receiving TARP packets. 
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• S40. A background loop operation is performed which applies an algorithm which 
determines the generation of decoy IP packets. The loop is interrupted when an 
encrypted TARP packet is received. 

5 • S42. The TARP packet may be probed to authenticate the packet before 
attempting to decrypt it using the link key. 

• S43. The TARP packet is decrypted with the appropriate link key to expose the 
destination TARP address and an indication of whether the packet is a decoy 
packet or part of a real message. 

10 • S44. If the packet is a decoy packet, the perishable decoy counter is incremented. 

• S45. Based on the decoy generation/dropping algorithm and the perishable decoy 
counter value, if the packet is a decoy packet, the receiver may choose to throw it 
away. 

• S46. The TARP packets are cached until all packets forming an interleave window 
15 are received. 

• S47. Once all packets of an interleave window are received, the packets are 
deinterleaved. 

• S48. The packets block of combined packets defining the interleave window is 
then decrypted using the session key. 

20 • S49. The decrypted block is then divided using the window sequence data and the 
IP r headers are converted into normal IP C headers. The window sequence 
numbers are integrated in the IP C headers. 

• S50. The packets are then handed up to the IP layer processes. 

L SCALABILITY ENHANCEMENTS 
25 The IP agility feature described above relies on the ability to transmit IP 

address changes to all TARP routers. The embodiments including this feature will be 
referred to as wi boutique^ embodiments due to potential limitations in scaling these 
features up for a large network, such as the Internet. (The "boutique" embodiments 
would, however, be robust for use in smaller networks, such as small virtual private 
30 networks, for example). One problem with the boutique embodiments is that if IP 
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address changes are to occur frequently, the message traffic required to update all 
routers sufficiently quickly creates a serious burden on the Internet when the TARP 
router and/or client population gets large. The bandwidth burden added to the 
networks, for example in 1CMP packets, that would be used to update all the TARP 
5 routers could overwhelm the Internet for a large scale implementation that approached 
the scale of the Internet. In other words, the boutique system's scalability is limited. 

A system can be constructed which trades some of the features of the above 
embodiments to provide the benefits of IP agility without the additional messaging 
burden. This is accomplished by IP address-hopping according to shared algorithms 
1 0 that govern IP addresses used between links participating in communications sessions 
between nodes such as TARP nodes. (Note that the IP hopping technique is also 
applicable to the boutique embodiment.) The IP agility feature discussed with respect 
to the boutique system can be modified so that it becomes decentralized under this 
scalable regime and governed by the above-described shared algorithm. Other 
1 5 features of the boutique system may be combined with this new type of IP-agility. 

The new embodiment has the advantage of providing IP agility governed by a 
local algorithm and set of IP addresses exchanged by each communicating pair of 
nodes. This local governance is session-independent in that it may govern 
communications between a pair of nodes, irrespective of the session or end points 
20 being transferred between the directly communicating pair of nodes. 

In the scalable embodiments, blocks of IP addresses are allocated to each node 
in the network. (This scalability will increase in the future, when Internet Protocol 
addresses are increased to 128-bit fields, vastly increasing the number of distinctly 
addressable nodes). Each node can thus use any of the IP addresses assigned to that 
25 node to communicate with other nodes in the network. Indeed, each pair of 
communicating nodes can use a plurality of source IP addresses and destination IP 
addresses for communicating with each other. 

Each communicating pair of nodes in a chain participating in any session 
stores two blocks of IP addresses, called netblocks, and an algorithm and 
30 randomization seed for selecting, from each netblock. the next pair of 
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source/destination IP addresses that will be used to transmit the next message. In 
other words, the algorithm governs the sequential selection of IP-address pairs, one 
sender and one receiver IP address, from each netblock. The combination of 
algorithm, seed, and netblock (IP address block) will be called a "hopblock." A router 
5 issues separate transmit and receive hopblocks to its clients. The send address and the 
receive address of the IP header of each outgoing packet sent by the client are filled 
with the send and receive IP addresses generated by the algorithm. The algorithm is 
"clocked" (indexed) by a counter so that each time a pair is used, the algorithm turns 
out a new transmit pair for the next packet to be sent. 

10 The router's receive hopblock is identical to the client's transmit hopblock. 

The router uses the receive hopblock to predict what the send and receive IP address 
pair for the next expected packet from that client will be. Since packets can be 
received out of order, it is not possible for the router to predict with certainty what IP 
address pair will be on the next sequential packet. To account for this problem, the 

15 router generates a range of predictions encompassing the number of possible 
transmitted packet send/receive addresses, of which the next packet received could 
leap ahead. Thus, if there is a vanishingly small probability that a given packet will 
arrive at the router ahead of 5 packets transmitted by the client before the given 
packet, then the router can generate a series of 6 send/receive IP address pairs (or 

20 "hop window") to compare with the next received packet. When a packet is received, 
it is marked in the hop window as such, so that a second packet with the same IP 
address pair will be discarded. If an out-of-sequence packet does not arrive within a 
predetermined timeout period, it can be requested for retransmission or simply 
discarded from the receive table, depending upon the protocol in use for that 

25 communications session, or possibly by convention. 

When the router receives the client's packet, it compares the send and receive 
IP addresses of the packet with the next N predicted send and receive IP address pairs 
and rejects the packet if it is not a member of this set. Received packets that do not 
have the predicted source/destination IP addresses falling with the window are 

30 rejected, thus thwarting possible hackers. (With the number of possible combinations, 
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even a fairly large window would be hard to fall into at random.) If it is a member of 
this set, the router accepts the packet and processes it further. This link-based IP- 
hopping strategy, referred to as "IHOP," is a network element that stands on its own 
and is not necessarily accompanied by elements of the boutique system described 
5 above. If the routing agility feature described in connection with the boutique 
embodiment is combined with this link-based IP-hopping strategy, the router's next 
step would be to decrypt the TARP header to determine the destination TARP router 
for the packet and determine what should be the next hop for the packet. The TARP 
router would then forward the packet to a random TARP router or the destination 

10 TARP router with which the source TARP router has a link-based IP hopping 
communication established. 

Figure 8 shows how a client computer 801 and a TARP router 811 can 
establish a secure session. When client 801 seeks to establish an IHOP session with 
TARP router 81 L the client 801 sends "secure synchronization*' request ("SSYN") 

15 packet 821 to the TARP router 811. This SYN packet 821 contains the client's 801 
authentication token, and may be sent to the router 81 1 in an encrypted format. The 
source and destination IP numbers on the packet 821 are the client's 801 current fixed 
IP address, and a "known" fixed IP address for the router 811. (For security purposes, 
it may be desirable to reject any packets from outside of the local network that are 

20 destined for the router's known fixed IP address.) Upon receipt and validation of the 
client's 801 SSYN packet 821, the router 811 responds by sending an encrypted 
"secure synchronization acknowledgment" ( U SSYN ACK") 822 to the client 801. 
This SSYN ACK 822 will contain the transmit and receive hopblocks that the client 
801 will use when communicating with the TARP router 811. The client 801 will 

25 acknowledge the TARP router* s 81 1 response packet 822 by generating an encrypted 
SSYN ACK ACK packet 823 which will be sent from the client's 801 fixed IP 
address and to the TARP router's 81 1 known fixed IP address. The client 801 will 
simultaneously generate a SSYN ACK ACK packet; this SSYN ACK packet, referred 
to as the Secure Session Initiation (SSI) packet 824, will be sent with the first 

30 {sender, receiver] IP pair in the client's transmit table 921 (FIG. 9), as specified in the 



25 



WO 01/61922 



r<~ft/uaui/u<»<tu 



transmit hopblock provided by the TARP router 81 1 in the SSYN ACK packet 822. 
The TARP router 81 1 will respond to the SSI packet 824 with an SSI ACK packet 
825, which will be sent with the first {sender, receiver} IP pair in the TARP router's 
transmit table 923. Once these packets have been successfully exchanged, the secure 
5 communications session is established, and all further secure communications 
between the client 801 and the TARP router 81 1 will be conducted via this secure 
session, as long as synchronization is maintained. If synchronization is lost, then the 
client 801 and TARP router 802 may re-establish the secure session by the procedure 
outlined in Figure 8 and described above. 

10 While the secure session is active, both the client 901 and TARP router 911 

(FIG. 9) will maintain their respective transmit tables 921, 923 and receive tables 922, 
924, as provided by the TARP router during session synchronization 822. It is 
important that the sequence of IP pairs in the client's transmit table 921 be identical to 
those in the TARP router's receive table 924; similarly, the sequence of IP pairs in the 

15 client's receive table 922 must be identical to those in the router's transmit table 923. 
This is required for the session synchronization to be maintained. The client 901 need 
maintain only one transmit table 921 and one receive table 922 during the course of 
the secure session. Each sequential packet sent by the client 901 will employ the next 
[send, receive) IP address pair in the transmit table, regardless of TCP or UDP 

20 session. The TARP router 91 1 will expect each packet arriving from the client 901 to 
bear the next IP address pair shown in its receive table. 

Since packets can arrive out of order, however, the router 91 1 can maintain a 
"look ahead" buffer in its receive table, and will mark previously-received IP pairs as 
invalid for future packets; any future packet containing an IP pair that is in the look- 

25 ahead buffer but is marked as previously received will be discarded. Communications 
from the TARP router 91 1 to the client 901 are maintained in an identical manner; in 
particular, the router 91 1 will select the next IP address pair from its transmit table 
923 when constructing a packet to send to the client 901, and the client 901 will 
maintain a look-ahead buffer of expected IP pairs on packets that it is receiving. Each 
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TARP router will maintain separate pairs of transmit and receive tables for each client 
that is currently engaged in a secure session with or through that TARP router. 

While clients receive their hopblocks from the first server linking them to the 
Internet, routers exchange hopblocks. When a router establishes a link-based IP- 
5 hopping communication regime with another router, each router of the pair exchanges 
its transmit hopblock. The transmit hopblock of each router becomes the receive 
hopblock of the other router. The communication between routers is governed as 
described by the example of a client sending a packet to the first router. 

While the above strategy works fine in the IP milieu, many local networks that 

10 are connected to the Internet are Ethernet systems. In Ethernet, the IP addresses of the 
destination devices must be translated into hardware addresses, and vice versa, using 
known processes ("address resolution protocol." and "reverse address resolution 
protocol")- However, if the link-based IP-hopping strategy is employed, the 
correlation process would become explosive and burdensome. An alternative to the 

15 link-based IP hopping strategy may be employed within an Ethernet network. The 
solution is to provide that the node linking the Internet to the Ethernet (call it the 
border node) use the link-based IP-hopping communication regime to communicate 
with nodes outside the Ethernet LAN. Within the Ethernet LAN, each TARP node 
would have a single IP address which would be addressed in the conventional way. 

20 Instead of comparing the {sender, receiver) IP address pairs to authenticate a packet, 
the intra-LAN TARP node would use one of the IP header extension fields to do so. 
Thus, the border node uses an algorithm shared by the intra-LAN TARP node to 
generate a symbol that is stored in the free field in the IP header, and the intra-LAN 
TARP node generates a range of symbols based on its prediction of the next expected 

25 packet to be received from that particular source IP address. The packet is rejected if 
it does not fall into the set of predicted symbols (for example, numerical values) or is 
accepted if it does. Communications from the intra-LAN TARP node to the border 
node are accomplished in the same manner, though the algorithm will necessarily be 
different for security reasons. Thus, each of the communicating nodes will generate 

30 transmit and receive tables in a similar manner to that of Figure 9; the intra-LAN 
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TARP nodes transmit table vill be identical to the border node's receive table, and the 
intra-LAN TARP node's receive table will be identical to the border node's transmit 
table. 

The algorithm used for IP address-hopping can be any desired algorithm. For 
5 example, the algorithm can be a given pseudo-random number generator that 
generates numbers of the range covering the allowed IP addresses with a given seed. 
Alternatively, the session participants can assume a certain type of algorithm and 
specify simply a parameter for applying the algorithm. For example the assumed 
algorithm could be a particular pseudo-random number generator and the session 
10 participants could simply exchange seed values. 

Note that there is no permanent physical distinction between the originating 
and destination terminal nodes. Either device at either end point can initiate a 
synchronization of the pair. Note also that the authentication/synchronization-request 
(and acknowledgment) and hopblock-exchange may all be served by a single message 
15 so that separate message exchanges may not be required. 

As another extension to the stated architecture, multiple physical paths can be 
used by a client, in order to provide link redundancy and further thwart attempts at 
denial of service and traffic monitoring. As shown in Figure 10, for example, client 
1001 can establish three simultaneous sessions with each of three TARP routers 
20 provided by different ISPs 101 1, 1012, 1013. As an example, the client 1001 can use 
three different telephone lines 1021, 1022, 1023 to connect to the ISPs, or two 
telephone lines and a cable modem, etc. In this scheme, transmitted packets will be 
sent in a random fashion among the different physical paths. This architecture 
provides a high degree of communications redundancy, with improved immunity 
25 from denial-of-service attacks and traffic monitoring. 

2. FURTHER EXTENSIONS 
The following describes various extensions to the techniques, systems, and 
methods described above. As described above, the security of communications 
occurring between computers in a computer network (such as the Internet, an 
30 Ethernet, or others) can be enhanced by using seemingly random source and 
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destination Internet Protocol (IP) addresses for data packets transmitted over the 
network. This feature prevents eavesdroppers from determining which computers in 
the network are communicating with each other while permitting the two 
communicating computers to easily recognize whether a given received data packet is 
legitimate or not. In one embodiment of the above-described systems, an IP header 
extension field is used to authenticate incoming packets on an Ethernet. 

Various extensions to the previously described techniques described herein 
include: (1) use of hopped hardware or "MAC addresses in broadcast type network; 
(2) a self-synchronization technique that permits a computer to automatically regain 
synchronization with a sender; (3) synchronization algorithms that allow transmitting 
and receiving computers to quickly re-establish synchronization in the event of lost 
packets or other events; and (4) a fast-packet rejection mechanism for rejecting 
invalid packets. Any or all of these extensions can be combined with the features 
described above in any of various ways. 

A. Hardware Address Hopping 

Internet protocol-based communications techniques on a LAN — or across any 
dedicated physical medium— typically embed the IP packets within lower-level 
packets, often referred to as "frames/' As shown in FIG. 1 1, for example, a first 
Ethernet frame 1150 comprises a frame header 1101 and two embedded IP packets 
IP1 and IP2. while a second Ethernet frame 1 160 comprises a different frame header 
1104 and a single IP packet IP3. Each frame header generally includes a source 
hardware address 1 101 A and a destination hardware address 1 101 B; other well- 
known fields in frame headers are omitted from FIG. 1 1 for clarity. Two hardware 
nodes communicating over a physical communication channel insert appropriate 
source and destination hardware addresses to indicate which nodes on the channel or 
network should receive the frame. 

It may be possible for a nefarious listener to acquire information about the 
contents of a frame and/or its communicants by examining frames on a local network 
rather than (or in addition to) the IP packets themselves. This is especially true in 
broadcast media, such as Ethernet, where it is necessary to insert into the frame 
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header the hardware address of the machine that generated the frame and the 
hardware address of the machine to which frame is being sent. All nodes on the 
network can potentially "see" all packets transmitted across the network. This can be 
a problem for secure communications, especially in cases where the communicants do 
5 not want for any third party to be able to identify who is engaging in the information 
exchange. One way to address this problem is to push the address-hopping scheme 
down to the hardware layer. In accordance with various embodiments of the 
invention, hardware addresses are "hopped" in a manner similar to that used to change 
IP addresses, such that a listener cannot determine which hardware node generated a 

1 0 particular message nor which node is the intended recipient. 

FIG. 12A shows a system in which Media Access Control ( U MAC") hardware 
addresses are "hopped" in order to increase security over a network such as an 
Ethernet. While the description refers to the exemplary case of an Ethernet 
environment, the inventive principles are equally applicable to other types of 

15 communications media. In the Ethernet case, the MAC address of the sender and 
receiver are inserted into the Ethernet frame and can be observed by anyone on the 
LAN who is within the broadcast range for that frame. For secure communications, it 
becomes desirable to generate frames with MAC addresses that are not attributable to 
any specific sender or receiver. 

20 As shown in FIG. 12 A, two computer nodes 1201 and 1202 communicate over 

a communication channel such as an Ethernet. Each node executes one or more 
application programs 1203 and 1218 that communicate by transmitting packets 
through communication software 1204 and 1217, respectively. Examples of 
application programs include video conferencing, e-mail, word processing programs, 

25 telephony, and the like. Communication software 1204 and 1217 can comprise, for 
example, an OSI layered architecture or "stack" that standardizes various services 
provided at different levels of functionality. 

The lowest levels of communication software 1204 and 1217 communicate 
with hardware components 1206 and 1214 respectively, each of which can include 

30 one or more registers 1207 and 1215 that allow the hardware to be reconfigured or 
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controlled in accordance with various communication protocols. The hardware 
components (an Ethernet network interface card, for example) communicate with 
each other over the communication medium. Each hardware component is typically 
pre-assigned a fixed hardware address or MAC number that identifies the hardware 
5 component to other nodes on the network. One or more interface drivers control the 
operation of each card and can, for example, be configured to accept or reject packets 
from certain hardware addresses. As will be described in more detail below, various 
embodiments of the inventive principles provide for "hopping" different addresses 
using one or more algorithms and one or more moving windows that track a range of 

10 valid addresses to validate received packets. Packets transmitted according to one or 
more of the inventive principles will be generally referred to as "secure" packets or 
"secure communications" to differentiate them from ordinary data packets that are 
transmitted in the clear using ordinary, machine-correlated addresses. 

One straightforward method of generating non-attributable MAC addresses is 

15 an extension of the IP hopping scheme. In this scenario, two machines on the same 
LAN that desire to communicate in a secure fashion exchange random-number 
generators and seeds, and create sequences of quasi-random MAC addresses for 
synchronized hopping. The implementation and synchronization issues are then 
similar to that of IP hopping. 

20 This approach, however, runs the risk of using MAC addresses that are 

currently active on the LAN — which, in turn, could interrupt communications for 
those machines. Since an Ethernet MAC address is at present 48 bits in length, the 
chance of randomly misusing an active MAC address is actually quite small. 
However, if that figure is multiplied by a large number of nodes (as would be found 

25 on an extensive LAN), by a large number of frames (as might be the case with packet 
voice or streaming video), and by a large number of concurrent Virtual Private 
Networks (VPNs), then the chance that a non-secure machine's MAC address could 
be used in an address-hopped frame can become non-trivial. In short, any scheme that 
runs even a small risk of interrupting communications for other machines on the LAN 

30 is bound to receive resistance from prospective system administrators. Nevertheless, it 



is technically feasible, and car be implemented without risk on a LAN on which there 
is a small number of machines, or if all of the machines on the LAN are engaging in 
MAC-hopped communications. 

Synchronized MAC address hopping may incur some overhead in the course 
5 of session establishment, especially if there are multiple sessions or multiple nodes 
involved in the communications. A simpler method of randomizing MAC addresses is 
to allow each node to receive and process every incident frame on the network. 
Typically, each network interface driver will check the destination MAC address in 
the header of every incident frame to see if it matches that machine's MAC address; if 

10 there is no match, then the frame is discarded. In one embodiment, however, these 
checks can be disabled, and every incident packet is passed to the TARP stack for 
processing. This will be referred to as "promiscuous" mode, since every incident 
frame is processed. Promiscuous mode allows the sender to use completely random, 
unsynchronized MAC addresses, since the destination machine is guaranteed to 

15 process the frame. The decision as to whether the packet was truly intended for that 
machine is handled by the TARP stack, which checks the source and destination IP 
addresses for a match in its IP synchronization tables. If no match is found, the packet 
is discarded; if there is a match, the packet is unwrapped, the inner header is 
evaluated, and if the inner header indicates that the packet is destined for that machine 

20 then the packet is forwarded to the IP stack — otherwise it is discarded. 

One disadvantage of purely-random MAC address hopping is its impact on 
processing overhead; that is, since every incident frame must be processed, the 
machine's CPU is engaged considerably more often than if the network interface 
driver is discriminating and rejecting packets unilaterally. A compromise approach is 

25 to select either a single fixed MAC address or a small number of MAC addresses 
(e.g.. one for each virtual private network on an Ethernet) to use for MAC-hopped 
communications, regardless of the actual recipient for which the message is intended. 
In this mode, the network interface driver can check each incident frame against one 
(or a few) pre-established MAC addresses, thereby freeing the CPU from the task of 

30 physical-layer packet discrimination. This scheme does not betray any useful 

32 



information to an interloper on the LAN; in particular, every secure packet can 
already be identified by a unique packet type in the outer header. However, since all 
machines engaged in secure communications would either be using the same MAC 
address, or be selecting from a small pool of predetermined MAC addresses, the 
5 association between a specific machine and a specific MAC address is effectively 
broken. 

In this scheme, the CPU will be engaged more often than it would be in non- 
secure communications (or in. synchronized MAC address hopping), since the 
network interface driver cannot always unilaterally discriminate between secure 

10 packets that are destined for that machine, and secure packets from other VPNs. 
However, the non-secure traffic is easily eliminated at the network interface, thereby 
reducing the amount of processing required of the CPU. There are boundary 
conditions where these statements would not hold, of course— e.g., if all of the traffic 
on the LAN is secure traffic, then the CPU would be engaged to the same degree as it 

15 is in the purely-random address hopping case; alternatively, if each VPN on the LAN 
uses a different MAC address, then the network interface can perfectly discriminate 
secure frames destined for the local machine from those constituting other VPNs. 
These are engineering tradeoffs that might be best handled by providing 
administrative options for the users when installing the software and/or establishing 

20 VPNs. 

Even in this scenario, however, there still remains a slight risk of selecting 
MAC addresses that are being used by one or more nodes on the LAN. One solution 
to this problem is to formally assign one address or a range of addresses for use in 
MAC-hopped communications. This is typically done via an assigned numbers 

25 registration authority; e.g.. in the case of Ethernet, MAC address ranges are assigned 
to vendors by the Institute of Electrical and Electronics Engineers (IEEE). A 
formally-assigned range of addresses would ensure that secure frames do not conflict 
with any properly-configured and properly-functioning machines on the LAN. 

Reference will now be made to FIGS. 12A and 12B in order to describe the 

30 many combinations and features that follow the inventive principles. As explained 
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above, two computer nodes 1201 and 1202 are assumed to be communicating over a 
network or communication medium such as an Ethernet. A communication protocol 
in each node (1204 and 1217, respectively) contains a modified element 1205 and 
1216 that performs certain functions that deviate from the standard communication 
protocols. In particular, computer node 1201 implements a first "hop" algorithm 
1208X that selects seemingly random source and destination IP addresses (and, in one 
embodiment, seemingly random IP header discriminator fields) in order to transmit 
each packet to the other computer node. For example, node 1201 maintains a transmit 
table 1208 containing triplets of source (S), destination (D), and discriminator fields 
(DS) that are inserted into outgoing IP packet headers. The table is generated through 
the use of an appropriate algorithm (e.g., a random number generator that is seeded 
with an appropriate seed) that is known to the recipient node 1202. As each new IP 
packet is formed, the next sequential entry out of the sender's transmit table 1208 is 
used to populate the IP source. IP destination, and IP header extension field (e.g., 
discriminator field). It will be appreciated that the transmit table need not be created 
in advance but could instead be created on-the-fly by executing the algorithm when 
each packet is formed. 

At the receiving node 1202, the same IP hop algorithm 1222X is maintained 
and used to generate a receive table 1222 that lists valid triplets of source IP address, 
destination IP address, and discriminator field. This is shown by virtue of the first 
five entries of transmit table 1208 matching the second five entries of receive table 
1222. (The tables may be slightly offset at any particular time due to lost packets, 
misordered packets, or transmission delays). Additionally, node 1202 maintains a 
receive window W3 that represents a list of valid IP source, IP destination, and 
discriminator fields that will be accepted when received as part of an incoming IP 
packet. As packets are received, window W3 slides down the list of valid entries, 
such that the possible valid entries change over time. Two packets that arrive out of 
order but are nevertheless matched to entries within window W3 will be accepted; 
those falling outside of window W3 will be rejected as invalid. The length of window 
W3 can be adjusted as necessary to reflect network delays or other factors. 
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Node 1202 maintains a similar transmit table 1221 for creating IP packets and 
frames destined for node 1201 using a potentially different hopping algorithm 122 IX, 
and node 1201 maintains a matching receive table 1209 using the same algorithm 
1209X. As node 1202 transmits packets to node 1201 using seemingly random IP 
source, IP destination, and/or discriminator fields, node 1201 matches the incoming 
packet values to those falling within window Wl maintained in its receive table. In 
effect, transmit table 1208 of node 1201 is synchronized (i.e., entries are selected in 
the same order) to receive table 1222 of receiving node 1202. Similarly, transmit 
table 1221 of node 1202 is synchronized to receive table 1209 of node 1201. It will 
be appreciated that although a common algorithm is shown for the source, destination 
and discriminator fields in FIG. 12A (using, e.g., a different seed for each of the three 
fields), an entirely different algorithm could in fact be used to establish values for 
each of these fields. It will also be appreciated that one or two of the fields can be 
"hopped" rather than all three as illustrated. 

In accordance with another aspect of the invention, hardware or "MAC" 
addresses are hopped instead of or in addition to IP addresses and/or the discriminator 
field in order to improve security in a local area or broadcast-type network. To that 
end, node 1201 further maintains a transmit table 1210 using a transmit algorithm 
121 OX to generate source and destination hardware addresses that are inserted into 
frame headers (e.g., fields 11 01 A and 1 101 B in FIG. 11) that are synchronized to a 
corresponding receive table 1224 at node 1202. Similarly, node 1202 maintains a 
different transmit table 1223 containing source and destination hardware addresses 
that is synchronized with a corresponding receive table 1211 at node 1201. In this 
manner, outgoing hardware frames appear to be originating from and going to 
completely random nodes on the network, even though each recipient can determine 
whether a given packet is intended for it or not. It will be appreciated that the 
hardware hopping feature can be implemented at a different level in the 
communications protocol than the IP hopping feature (e.g., in a card driver or in a 
hardware card itself to improve performance). 
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FIG. 12B shows three different embodiments or modes that can be employed 
using the aforementioned principles. In a first mode referred to as "promiscuous" 
mode, a common hardware address (e.g., a fixed address for source and another for 
destination) or else a completely random hardware address is used by all nodes on the 
5 network, such that a particular packet cannot be attributed to any one node. Each 
node must initially accept all packets containing the common (or random) hardware 
address and inspect the IP addresses or discriminator field to determine whether the 
packet is intended for that node. In this regard, either the IP addresses or the 
discriminator field or both can be varied in accordance with an algorithm as described 

10 above. As explained previously, this may increase each node's overhead since 
additional processing is involved to determine whether a given packet has valid 
source and destination hardware addresses. 

In a second mode referred to as "promiscuous per VPN" mode, a small set of 
fixed hardware addresses are used, with a fixed source/destination hardware address 

15 used for all nodes communicating over a virtual private network. For example, if 
there are six nodes on an Ethernet, and the network is to be split up into two private 
virtual networks such that nodes on one VPN can communicate with only the other 
two nodes on its own VPN, then two sets of hardware addresses could be used: one 
set for the first VPN and a second set for the second VPN. This would reduce the 

20 amount of overhead involved in checking for valid frames since only packets arriving 
from the designated VPN would need to be checked. IP addresses and one or more 
discriminator fields could still be hopped as before for secure communication within 
the VPN. Of course, this solution compromises the anonymity of the VPNs (i.e., an 
outsider can easily tell what traffic belongs in which VPN, though he cannot correlate 

25 it to a specific machine/person). It also requires the use of a discriminator field to 
mitigate the vulnerability to certain types of DoS attacks. (For example, without the 
discriminator field, an attacker on the LAN could stream frames containing the MAC 
addresses being used by the VPN; rejecting those frames could lead to excessive 
processing overhead. The discriminator field would provide a low-overhead means of 

30 rejecting the false packets.) 
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In a third mode referred to as "hardware hopping" mode, hardware addresses 
are varied as illustrated in FIG. 12 A, such that hardware source and destination 
addresses are changed constantly in order to provide non-attributable addressing. 
Variations on these embodiments are of course possible, and the invention is not 
5 intended to be limited in any respect by these illustrative examples. 

B. Extending the Address Space 
Address hopping provides security and privacy. However, the level of 
protection is limited by the number of addresses in the blocks being hopped. A 
hopblock denotes a field or fields modulated on a packet- wise basis for the purpose of 

10 providing a VPN. For instance, if two nodes communicate with IP address hopping 
using hopblocks of 4 addresses (2 bits) each, there would be 16 possible address-pair 
combinations. A window of size 16 would result in most address pairs being accepted 
as valid most of the time. This limitation can be overcome by using a discriminator 
field in addition to or instead of the hopped address fields. The discriminator field 

15 would be hopped in exactly the same fashion as the address fields and it would be 
used to determine whether a packet should be processed by a receiver. 

Suppose that two clients, each using four-bit hopblocks, would like the same 
level of protection afforded to clients communicating via IP hopping between two A 
blocks (24 address bits eligible for hopping). A discriminator field of 20 bits, used in 

20 conjunction with the 4 address bits eligible for hopping in the IP address field, 
provides this level of protection. A 24-bit discriminator field would provide a similar 
level of protection if the address fields were not hopped or ignored. Using a 
discriminator field offers the following advantages: (1) an arbitrarily high level of 
protection can be provided, and (2) address hopping is unnecessary to provide 

25 protection. This may be important in environments where address hopping would 
cause routing problems. 
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C. Synchronization Techniques 

It is generally assumed that once a sending node and receiving node have 
exchanged algorithms and seeds (or similar information sufficient to generate quasi- 
random source and destination tables), subsequent communication between the two 
5 nodes will proceed smoothly. Realistically, however, two nodes may lose 
synchronization due to network delays or outages, or other problems. Consequently, 
it is desirable to provide means for re-establishing synchronization between nodes in a 
network that have lost synchronization. 

One possible technique is to require that each node provide an 

10 acknowledgment upon successful receipt of each packet and, if no acknowledgment is 
received within a certain period of time, to re-send the unacknowledged packet. This 
approach, however, drives up overhead costs and may be prohibitive in high- 
throughput environments such as streaming video or audio, for example. 

A different approach is to employ an automatic synchronizing technique that 

15 will be referred to herein as "self-synchronization." In this approach, synchronization 
information is embedded into each packet, thereby enabling the receiver to re- 
synchronize itself upon receipt of a single packet if it determines that is has lost 
synchronization with the sender. (If communications are already in progress, and the 
receiver determines that it is still in sync with the sender, then there is no need to re- 

20 synchronize.) A receiver could detect that it was out of synchronization by, for 
example, employing a "dead-man" timer that expires after a certain period of time, 
wherein the timer is reset with each valid packet. A time stamp could be hashed into 
the public sync field (see below) to preclude packet-retry attacks. 

In one embodiment, a "sync field" is added to the header of each packet sent 

25 out by the sender. This sync field could appear in the clear or as part of an encrypted 
portion of the packet. Assuming that a sender and receiver have selected a random- 
number generator (RNG) and seed value, this combination of RNG and seed can be 
used to generate a random-number sequence (RNS). The RNS is then used to generate 
a sequence of source/destination IP pairs (and. if desired, discriminator fields and 

30 hardware source and destination addresses), as described above. It is not necessary, 



38 



WU U1/61V2Z 



however, to generate the entire sequence (or the first N-l values) in order to generate 
the Nth random number in the sequence; if the sequence index N is known, the 
random value corresponding to that index can be directly generated (see below). 
Different RNGs (and seeds) with different fundamental periods could be used to 
generate the source and destination IP sequences, but the basic concepts would still 
apply. For the sake of simplicity, the following discussion will assume that IP source 
and destination address pairs (only) are hopped using a single RNG sequencing 
mechanism. 

In accordance with a "self-synchronization"" feature, a sync field in each 
packet header provides an index (i.e., a sequence number) into the RNS that is being 
used to generate IP pairs. Plugging this index into the RNG that is being used to 
generate the RNS yields a specific random number value, which in turn yields a 
specific IP pair. That is, an IP pair can be generated directly from knowledge of the 
RNG, seed, and index number; it is not necessary, in this scheme, to generate the 
entire sequence of random numbers that precede the sequence value associated with 
the index number provided. 

Since the communicants have presumably previously exchanged RNGs and 
seeds, the only new information that must be provided in order to generate an IP pair 
is the sequence number. If this number is provided by the sender in the packet header, 
then the receiver need only plug this number into the RNG in order to generate an IP 
pair - and thus verify that the IP pair appearing in the header of the packet is valid. In 
this scheme, if the sender and receiver lose synchronization, the receiver can 
immediately re-synchronize upon receipt of a single packet by simply comparing the 
IP pair in the packet header to the IP pair generated from the index number. Thus, 
synchronized communications can be resumed upon receipt of a single packet, 
making this scheme ideal for multicast communications. Taken to the extreme, it 
could obviate the need for synchronization tables entirely; that is, the sender and 
receiver could simply rely on the index number in the sync field to validate the IP pair 
on each packet, and thereby eliminate the tables entirely. 
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The aforementioned scheme may have some inherent security issues 
associated with it — namely, the placement of the sync field. If the field is placed in 
the outer header, then an interloper could observe the values of the field and their 
relationship to the IP stream. This could potentially compromise the algorithm that is 
5 being used to generate the IP-address sequence, which would compromise the security 
of the communications. If, however, the value is placed in the inner header, then the 
sender must decrypt the inner header before it can extract the sync value and validate 
the IP pair; this opens up the receiver to certain types of denial-of-service (DoS) 
attacks, such as packet replay. That is, if the receiver must decrypt a packet before it 

10 can validate the IP pair, then it could potentially be forced to expend a significant 
amount of processing on decryption if an attacker simply retransmits previously valid 
packets. Other attack methodologies are possible in this scenario. 

A possible compromise between algorithm security and processing speed is to 
split up the sync value between an inner (encrypted) and outer (unencrypted) header. 

15 That is, if the sync value is sufficiently long, it could potentially be split into a 
rapidly-changing part that can be viewed in the clear, and a fixed (or very slowly 
changing) part that must be protected. The part that can be viewed in the clear will be 
called the "public sync" portion and the part that must be protected will be called the 
"private sync" portion. 

20 Both the public sync and private sync portions are needed to generate the 

complete sync value. The private portion, however, can be selected such that it is 
fixed or will change only occasionally. Thus, the private sync value can be stored by 
the recipient, thereby obviating the need to decrypt the header in order to retrieve it. If 
the sender and receiver have previously agreed upon the frequency with which the 
private part of the sync will change, then the receiver can selectively decrypt a single 
header in order to extract the new private sync if the communications gap that has led 
to lost synchronization has exceeded the lifetime of the. previous private sync. This 
should not represent a burdensome amount of decryption, and thus should not open up 
the receiver to denial-of-service attack simply based on the need to occasionally 
decrypt a single header. 
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One implementation of this is to use a hashing function with a one-to-one 
mapping to generate the private and public sync portions from the sync value. This 
implementation is shown in FIG. 13, where (for example) a first ISP 1302 is the 
sender and a second ISP 1303 is the receiver. (Other alternatives are possible from 
5 FIG. 13.) A transmitted packet comprises a public or "outer" header 1305 that is not 
encrypted, and a private or "inner" header 1306 that is encrypted using for example a 
link key. Outer header 1305 includes a public sync portion while inner header 1306 
contains the private sync portion. A receiving node decrypts the inner header using a 
decryption function 1307 in order to extract the private sync portion. This step is 

10 necessary only if the lifetime of the currently buffered private sync has expired. (If 
the currently-buffered private sync is still valid, then it is simply extracted from 
memory and "added" (which could be an inverse hash) to the public sync, as shown in 
step 1308.) The public and decrypted private sync portions are combined in function 
1308 in order to generate the combined sync 1309. The combined sync (1309) is then 

15 fed into the RNG (1310) and compared to the IP address pair (131 1) to validate or 
reject the packet. 

An important consideration in this architecture is the concept of "future" and 
"past" where the public sync values are concerned. Though the sync values, 
themselves, should be random to prevent spoofing attacks, it may be important that 

20 the receiver be able to quickly identify a sync value that has already been sent — even 
if the packet containing that sync value was never actually received by the receiver. 
One solution is to hash a time stamp or sequence number into the public sync portion, 
which could be quickly extracted, checked, and discarded, thereby validating the 
public sync portion itself. 

25 In one embodiment, packets can be checked by comparing the 

source/destination IP pair generated by the sync field with the pair appearing in the 
packet header. If (1) they match. (2) the time stamp is valid, and (3) the dead-man 
timer has expired, then re-synchronization occurs; otherwise, the packet is rejected. If 
enough processing power is available, the dead-man timer and synchronization tables 
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can be avoided altogether, and the receiver would simply resynchronize (e.g., 
validate) on every packet. 

The foregoing scheme may require large-integer (e.g., 160-bit) math, which 
may affect its implementation. Without such large-integer registers, processing 
5 throughput would be affected, thus potentially affecting security from a denial-of- 
service standpoint. Nevertheless, as large-integer math processing features become 
more prevalent the costs of implementing such a feature will be reduced. 

D. Other Synchronization Schemes 
As explained above, if W or more consecutive packets are lost between a 
10 transmitter and receiver in a VPN (where W is the window size), the receivers 
window will not have been updated and the transmitter will be transmitting packets 
not in the receiver's window. The sender and receiver will not recover 
synchronization until perhaps the random pairs in the window are repeated by chance. 
Therefore, there is a need to keep a transmitter and receiver in synchronization 
1 5 whenever possible and to re-establish synchronization whenever it is lost. 

A "checkpoint" scheme can be used to regain synchronization between a 
sender and a receiver that have fallen out of synchronization. In this scheme, a 
checkpoint message comprising a random IP address pair is used for communicating 
synchronization information. In one embodiment, two messages are used to 
20 communicate synchronization information between a sender and a recipient: 

1. SYNCREQ is a message used by the sender to indicate that it wants to 
synchronize; and 

2. SYNCACK is a message used by the receiver to inform the transmitter 
that it has been synchronized. 

25 According to one variation of this approach, both the transmitter and receiver 
maintain three checkpoints (see FIG. 14): 

1 . In the transmitter, ckpt_o ("checkpoint old") is the IP pair that was used to 
re-send the last SYNC REQ packet to the receiver. In the receiver, 
ckpt_o ("checkpoint old") is the IP pair that receives repeated SYNC REQ 
30 packets from the transmitter. 
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2. In the transmitter, ckptji ("checkpoint new") is the IP pair that will be 
used to send the next SYNC REQ packet to the receiver. In the receiver, 
ckpt_n ("checkpoint new") is the IP pair that receives a new SYNCJREQ 
packet from the transmitter and which causes the receiver's window to be 

5 re-aligned, ckpt_o set to ckpt_n, a new ckptn to be generated and a new 

ckpt_r to be generated. 

3. In the transmitter, ckpt_r is the IP pair that will be used to send the next 
SYNCACK packet to the receiver. In the receiver, ckpt_r is the IP pair 
that receives a new SYMC_ACK packet from the transmitter and which 

10 causes a new ckpt n to be generated. Since SYNC ACK is transmitted 

from the receiver ISP to the sender ISP. the transmitter ckptr refers to the 
ckpt_r of the receiver and the receiver ckpt r refers to the ckpt_r of the 
transmitter (see FIG. 14). 
When a transmitter initiates synchronization, the IP pair it will use to transmit the next 
15 data packet is set to a predetermined value and when a receiver first receives a 
SYNCJIEQ, the receiver window is updated to be centered on the transmitter's next 
IP pair. This is the primary mechanism for checkpoint synchronization. 

Synchronization can be initiated by a packet counter (e.g., after every N 
packets transmitted, initiate a synchronization) or by a timer (every S seconds, initiate 
20 a synchronization) or a combination of both. See FIG. 15. From the transmitter's 
perspective, this technique operates as follows: (1) Each transmitter periodically 
transmits a "sync request" message to the receiver to make sure that it is in sync. (2) 
If the receiver is still in sync, it sends back a "sync ack" message. (If this works, no 
further action is necessary). (3) If no "sync ack" has been received within a period of 
25 time, the transmitter retransmits the sync request again. If the transmitter reaches the 
next checkpoint without receiving a "sync ack" response, then synchronization is 
broken, and the transmitter should stop transmitting. The transmitter will continue to 
send sync_reqs until it receives a sync_ack , at which point transmission is 
reestablished. 
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From the receiver's perspective, the scheme operates as follows: (1) when it 
receives a "sync request" recuest from the transmitter, it advances its window to the 
next checkpoint position (even skipping pairs if necessary), and sends a "sync ack" 
message to the transmitter. If sync was never lost, then the "jump ahead" really just 
5 advances to the next available pair of addresses in the table (i.e., normal 
advancement). 

If an interloper intercepts the "sync request" messages and tries to interfere 
with communication by sending new ones, it will be ignored if the synchronization 
has been established or it it will actually help to re-establish synchronization. 

10 A window is realigned whenever a re-synchronization occurs. This 

realignment entails updating the receiver's window to straddle the address pairs used 
by the packet transmitted immediately after the transmission of the SYNCREQ 
packet. Normally, the transmitter and receiver are in synchronization with one 
another. However, when network events occur, the receiver's window may have to be 

15 advanced by many steps during resynchronization. In this case, it is desirable to move 
the window ahead without having to step through the intervening random numbers 
sequentially. (This feature is also desirable for the auto-sync approach discussed 
above). 

E. Random Number Generator with a Jump-Ahead capability 

20 An attractive method for generating randomly hopped addresses is to use 

identical random number generators in the transmitter and receiver and advance them 
as packets are transmitted and received. There are many random number generation 
algorithms that could be used. Each one has strengths and weaknesses for address 
hopping applications. 

25 Linear congruential random number generators (LCRs) are fast, simple and 

well characterized random number generators that can be made to jump ahead n steps 
efficiently. An LCR generates random numbers X], X 2 , X 3 ... X k starting with seed X 0 
using a recurrence 

Xi=(aX M + b)modc, (1) 

30 where a ? b and c define a particular LCR. Another expression for Xj, 
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X i =((a i (Xo+b)-b)/(a- 1 )) mod c (2) 
enables the jump-ahead capability. The factor a' can grow very large even for modest i 
if left unfettered. Therefore some special properties of the modulo operation can be 
used to control the size and processing time required to compute (2). (2) can be 
5 rewritten as: 

XKa 1 (X 0 (a- 1 )+b)-b)/(a- 1 ) mod c. (3) 
It can be shown that: 

(a^a-n+bH^a-l) mod c = 

((a j mod((a-l )c)(X 0 (a-l )+b) *b) /(a-1)) mod c (4). 
10 (X 0 (a-l)+b) can be stored as (X () (a-l)+b) mod c, b as b mod c and compute a' 
mod((a-l )c) (this requires 0(log(i)) steps). 

A practical implementation of this algorithm would jump a fixed distance, n, 
between synchronizations; this is tantamount to synchronizing every n packets. The 
window would commence n IP pairs from the start of the previous window. Using 
1 5 Xj w , the random number at the j lh checkpoint, as X 0 and n as j, a node can store a 11 
mod((a-l )c) once per LCR and set 

Xj + , w =X nli+ i)=((a n mod((a-l )c) (X/ v (a-1 )+b)-b)/(a-l ))mod c ? (5) 
to generate the random number for the j+l lh synchronization. Using this construction, 
a node could jump ahead an arbitrary (but fixed) distance between synchronizations in 
20 a constant amount of time (independent of n). 

Pseudo-random number generators, in general, and LCRs, in particular, will 
eventually repeat their cycles. This repetition may present vulnerability in the IP 
hopping scheme. An adversary would simply have to wait for a repeat to predict 
future sequences. One way of coping with this vulnerability is to create a random 
25 number generator with a known long cycle. A random sequence can be replaced by a 
new random number generator before it repeats. LCRs can be constructed with known 
long cycles. This is not currently true of many random number generators. 

Random number generators can be cryptographically insecure. An adversary 
can derive the RNG parameters by examining the output or part of the output. This is 
30 true of LCGs. This vulnerability can be mitigated by incorporating an encryptor, 
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designed to scramble the output as part of the random number generator. The random 
number generator prevents an adversary from mounting an attack — e.g., a known 
plaintext attack — against the encryptor. 

F. Random Number Generator Example 
5 Consider a RNG where a=31,b=4 and c=15. For this case equation (1) 

becomes: 

Xj=(31 X M +4) mod 15. (6) 

If one sets Xo=l, equation (6) will produce the sequence 1, 5, 9, 13, 2, 6, 10, 
14. 3, 7, 11.0, 4, 8, 12. This sequence will repeat indefinitely. For a jump ahead of 3 
10 numbers in this sequence a n = 31 3 =29791, c*(a-l )=1 5*30-450 and a n mod((a-l)c) = 
31 3 mod(15*30)=29791mod(450)=91. Equation (5) becomes: 

((91 (Xj30+4)-4)/30)mod 15 (7). 
Table 1 shows the jump ahead calculations from (7) . The calculations start at 5 and 
jump ahead 3. 
15 TABLE 1 
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G. Fast Packet Filter 

Address hopping VPNs must rapidly determine whether a packet has a valid 
header and thus requires further processing, or has an invalid header (a hostile packet) 
20 and should be immediately rejected. Such rapid determinations will be referred to as 
"fast packet filtering." This capability protects the VPN from attacks by an adversary 
who streams hostile packets at the receiver at a high rate of speed in the hope of 
saturating the receiver's processor (a so-called "denial of service" attack). Fast packet 
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filtering is an important feature for implementing VPNs on shared media such as 
Ethernet. 

Assuming that all participants in a VPN share an unassigned "A" block of 
addresses, one possibility is to use an experimental U A" block that will never be 
5 assigned to any machine that is not address hopping on the shared medium. "A" 
blocks have a 24 bits of address that can be hopped as opposed to the 8 bits in "C" 
blocks. In this case a hopblock will be the U A" block. The use of the experimental 
"A" block is a likely option on an Ethernet because: 

1 . The addresses have no validity outside of the Ethernet and will not be routed out 
10 to a valid outside destination by a gateway. 

2. There are 2 24 (-16 million) addresses that can be hopped within each "A" block. 
This yields >280 trillion possible address pairs making it very unlikely that an 
adversary would guess a valid address. It also provides acceptably low probability 
of collision between separate VPNs (all VPNs on a shared medium independently 

15 generate random address pairs from the same hi A" block). 

3. The packets will not be received by someone on the Ethernet who is not on a VPN 
(unless the machine is in promiscuous mode) minimizing impact on non-VPN 
computers. 

The Ethernet example will be used to describe one implementation of fast 
20 packet filtering. The ideal algorithm would quickly examine a packet header, 
determine whether the packet is hostile, and reject any hostile packets or determine 
which active IP pair the packet header matches. The problem is a classical associative 
memory problem. A variety of techniques have been developed to solve this problem 
(hashing. B-trees etc). Each of these approaches has its strengths and weaknesses. For 
25 instance, hash tables can be made to operate quite fast in a statistical sense, but can 
occasionally degenerate into a much slower algorithm. This slowness can persist for a 
period of time. Since there is a need to discard hostile packets quickly at all times, 
hashing would be unacceptable. 

H. Pr esence Vector Algorithm 
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A presence vector is a bit vector of length 2 n that can be indexed by w-bit 
numbers (each ranging fron 0 to 2 n -l). One can indicate the presence of k w-bit 
numbers (not necessarily unique), by setting the bits in the presence vector indexed by 
each number to 1 . Otherwise, the bits in the presence vector are 0. An rt-bit number, jc, 
5 is one of the k numbers if and only if the x th bit of the presence vector is 1. A fast 
packet filter can be implemented by indexing the presence vector and looking for a 1, 
which will be referred to as the "test." 

For example, suppose one wanted to represent the number 135 using a 
presence vector. The 135 th bit of the vector would be set. Consequently, one could 

1 0 very quickly determine whether an address of 1 35 was valid by checking only one bit: 
the 135 th bit. The presence vectors could be created in advance corresponding to the 
table entries for the IP addresses. In effect, the incoming addresses can be used as 
indices into a long vector, making comparisons very fast. As each RNG generates a 
new address, the presence vector is updated to reflect the information. As the window 

1 5 moves, the presence vector is updated to zero out addresses that are no longer valid. 

There is a trade-off between efficiency of the test and the amount of memory 
required for storing the presence vector(s). For instance, if one were to use the 48 bits 
of hopping addresses as an index, the presence vector would have to be 35 terabytes. 
Clearly, this is too large for practical purposes. Instead, the 48 bits can be divided into 

20 several smaller fields. For instance, one could subdivide the 48 bits into four 12-bit 
fields (see FIG. 16). This reduces the storage requirement to 2048 bytes at the expense 
of occasionally having to process a hostile packet. In effect, instead of one long 
presence vector, the decomposed address portions must match all four shorter 
presence vectors before further processing is allowed. (If the first part of the address 

25 portion doesn't match the first presence vector, there is no need to check the 
remaining three presence vectors). 

A presence vector will have a 1 in the y th bit if and only if one or more 
addresses with a corresponding field of y are active. An address is active only if each 
presence vector indexed by the appropriate sub-field of the address is 1 . 
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Consider a window of 32 active addresses and 3 checkpoints. A hostile packet 
will be rejected by the indexing of one presence vector more than 99% of the time. A 
hostile packet will be rejected by the indexing of all 4 presence vectors more than 
99.9999995% of the time. On average, hostile packets will be rejected in less than 
5 1 .02 presence vector index operations. 

The small percentage of hostile packets that pass the fast packet filter will be 
rejected when matching pairs are not found in the active window or are active 
checkpoints. Hostile packets that serendipitously match a header will be rejected 
when the VPN software attempts to decrypt the header. However, these cases will be 
10 extremely rare. There are many other ways this method can be configured to arbitrate 
the space/speed tradeoffs. 

I. Further Synchronization Enhancements 
A slightly modified form of the synchronization techniques described above 
can be employed. The basic principles of the previously described checkpoint 
15 synchronization scheme remain unchanged. The actions resulting from the reception 
of the checkpoints are, however, slightly different. In this variation, the receiver will 
maintain between OoO ("Out of Order") and 2xWINDOW_SIZE+OoO active 
addresses (1 <OoO <WINDOW_SIZE and WINDOWJSIZE >1). OoO and 
W1ND0W_S1ZE are engineerable parameters, where OoO is the minimum number of 
20 addresses needed to accommodate lost packets due to events in the network or out of 
order arrivals and WINDOWS1ZE is the number of packets transmitted before a 
SYNCREQ is issued. FIG. 17 depicts a storage array for a receiver's active 
addresses. 

The receiver starts with the first 2xWINDOW_SIZE addresses loaded and 
25 active (ready to receive data). As packets are received, the corresponding entries are 
marked as "used v and are no longer eligible to receive packets. The transmitter 
maintains a packet counter, initially set to 0. containing the number of data packets 
transmitted since the last initial transmission of a SYNCREQ for which 
SYNCACK has been received. When the transmitter packet counter equals 
30 WINDOW SIZE, the transmitter generates a SYNC REQ and does its initial 
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transmission. When the receiver receives a SYNC_REQ corresponding to its current 
CKPTN, it generates the next WINDOW_SIZE addresses and starts loading them in 
order starting at the first location after the last active address wrapping around to the 
beginning of the array after the end of the array has been reached. The receiver's array 
might look like FIG. 18 when a SYNCREQ has been received. In this case a couple 
of packets have been either lost or will be received out of order when the SYNC REQ 
is received. 

FIG. 19 shows the receivers array after the new addresses have been 
generated. If the transmitter does not receive a SYNC_ACK, it will re-issue the 
SYNC REQ at regular intervals. When the transmitter receives a SYNCACK, the 
packet counter is decremented by WINDOW_SIZE. If the packet counter reaches 
2x WINDOW JSIZE - OoO then the transmitter ceases sending data packets until the 
appropriate SYNC ACK is finally received. The transmitter then resumes sending 
data packets. Future behavior is essentially a repetition of this initial cycle. The 
advantages of this approach are: 

1 . There is no need for an efficient jump ahead in the random number generator, 

2. No packet is ever transmitted that does not have a corresponding entry in the 
receiver side 

3. No timer based re-synchronization is necessary. This is a consequence of 2. 

4. The receiver will always have the ability to accept data messages transmitted 
within OoO messages of the most recently transmitted message. 

J. Distributed Transmission Path Variant 
Another embodiment incorporating various inventive principles is shown in 
FIG. 20. In this embodiment, a message transmission system includes a first 
computer 2001 in communication with a second computer 2002 through a network 
2011 of intermediary computers. In one variant of this embodiment, the network 
includes two edge routers 2003 and 2004 each of which is linked to a plurality of 
Internet Service Providers (ISPs) 2005 through 2010. Each ISP is coupled to a 
plurality of other ISPs in an arrangement as shown in FIG. 20, which is a 
representative configuration only and is not intended to be limiting. Each connection 
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between ISPs is labeled in FIG. 20 to indicate a specific physical transmission path 
(e.g., AD is a physical path that links ISP A (element 2005) to ISP D (element 2008)). 
Packets arriving at each edge router are selectively transmitted to one of the ISPs to 
which the router is attached on the basis of a randomly or quasi-randomly selected 
5 basis. 

As shown in FIG. 21, computer 2001 or edge router 2003 incorporates a 
plurality of link transmission tables 2100 that identify, for each potential transmission 
path through the network, valid sets of IP addresses that can be used to transmit the 
packet. For example. AD table 2101 contains a plurality of IP source/destination 

10 pairs that are randomly or quasi-randomly generated. When a packet is to be 
transmitted from first computer 2001 to second computer 2002, one of the link tables 
is randomly (or quasi-randomly) selected, and the next valid source/destination 
address pair from that table is used to transmit the packet through the network. If path 
AD is randomly selected, for example, the next source/destination IP address pair 

15 (which is pre-determined to transmit between ISP A (element 2005) and ISP B 
(element 2008)) is used to transmit the packet. If one of the transmission paths 
becomes degraded or inoperative, that link table can be set to a "down" condition as 
shown in table 2105, thus preventing addresses from being selected from that table. 
Other transmission paths would be unaffected by this broken link. 

20 

3. CONTINUATION-IN-PART IMPROVEMENTS 

The following describes various improvements and features that can be 
applied to the embodiments described above. The improvements include: (1) a load 
balancer that distributes packets across different transmission paths according to 

25 transmission path quality; (2) a DNS proxy server that transparently creates a virtual 
private network in response to a domain name inquiry; (3) a large-to-small link 
bandwidth management feature that prevents denial-of-service attacks at system 
chokepoints; (4) a traffic limiter that regulates incoming packets by limiting the rate at 
which a transmitter can be synchronized with a receiver; and (5) a signaling 

30 synchronizer that allows a large number of nodes to communicate with a central node 
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by partitioning the communication function between two separate entities. Each is 
discussed separately below. 

A. Load Balancer 

Various embodiments described above include a system in which a 
5 transmitting node and a receiving node are coupled through a plurality of transmission 
paths, and wherein successive packets are distributed quasi-randomly over the 
plurality of paths. See, for example, FIGS. 20 and 21 and accompanying description. 
The improvement extends this basic concept to encompass distributing packets across 
different paths in such a manner that the loads on the paths are generally balanced 

1 0 according to transmission link quality. 

In one embodiment, a system includes a transmitting node and a receiving 
node that are linked via a plurality of transmission paths having potentially varying 
transmission quality. Successive packets are transmitted over the paths based on a 
weight value distribution function for each path. The rate that packets will be 

15 transmitted over a given path can be different for each path. The relative "health" of 
each transmission path is monitored in order to identify paths that have become 
degraded. In one embodiment, the health of each path is monitored in the transmitter 
by comparing the number of packets transmitted to the number of packet 
acknowledgements received. Each transmission path may comprise a physically 

20 separate path (e.g., via dial-up phone line, computer network, router, bridge, or the 
like), or may comprise logically separate paths contained within a broadband 
communication medium (e.g., separate channels in an FDM, TDM, CDMA, or other 
type of modulated or unmodulated transmission link). 

When the transmission quality of a path falls below a predetermined threshold 

25 and there are other paths that can transmit packets, the transmitter changes the weight 
value used for that path, making it less likely that a given packet will be transmitted 
over that path. The weight will preferably be set no lower than a minimum value that 
keeps nominal traffic on the path. The weights of the other available paths are altered 
to compensate for the change in the affected path. When the quality of a path degrades 

30 to where the transmitter is turned off by the synchronization function (i.e., no packets 
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are arriving at the destination), the weight is set to zero. If all transmitters are turned 
off. no packets are sent. 

Conventional TCP/IP protocols include a "throttling" feature that reduces the 
transmission rate of packets when it is determined that delays or errors are occurring 
5 in transmission. In this respect, timers are sometimes used to determine whether 
packets have been received. These conventional techniques for limiting transmission 
of packets, however, do not involve multiple transmission paths between two nodes 
wherein transmission across a particular path relative to the others is changed based 
on link quality. 

10 According to certain embodiments, in order to damp oscillations that might 

otherwise occur if weight distributions are changed drastically (e.g., according to a 
step function), a linear or an exponential decay formula can be applied to gradually 
decrease the weight value over time that a degrading path will be used. Similarly, if 
the health of a degraded path improves, the weight value for that path is gradually 

15 increased. 

Transmission link health can be evaluated by comparing the number of 
packets that are acknowledged within the transmission window (see embodiments 
discussed above) to the number of packets transmitted within that window and by the 
state of the transmitter (i.e., on or off). In other words, rather than accumulating 
20 general transmission statistics over time for a path, one specific implementation uses 
the "windowing" concepts described above to evaluate transmission path health. 

The same scheme can be used to shift virtual circuit paths from an '"unhealthy" 
path to a "healthy" one, and to select a path for a new virtual circuit. 

FIG. 22A shows a flowchart for adjusting weight values associated with a 
25 plurality of transmission links. It is assumed that software executing in one or more 
computer nodes executes the steps shown in FIG. 22A. It is also assumed that the 
software can be stored on a computer-readable medium such as a magnetic or optical 
disk for execution by a computer. 

Beginning in step 2201, the transmission quality of a given transmission path 
30 is measured. As described above, this measurement can be based on a comparison 
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between the number of packets transmitted over a particular link to the number of 
packet acknowledgements received over the link (e.g., per unit time, or in absolute 
terms). Alternatively, the quality can be evaluated by comparing the number of 
packets that are acknowledged within the transmission window to the number of 
5 packets that were transmitted within that window. In yet another variation, the 
number of missed synchronization messages can be used to indicate link quality. 
Many other variations are of course possible. 

In step 2202, a check is made to determine whether more than one transmitter 
(e.g., transmission path) is turned on. If not, the process is terminated and resumes at 
10 step 2201. 

In step 2203, the link quality is compared to a given threshold (e.g., 50%, or 
any arbitrary number). If the quality falls below the threshold, then in step 2207 a 
check is made to determine whether the weight is above a minimum level (e.g., 1%). 
If not, then in step 2209 the weight is set to the minimum level and processing 

15 resumes at step 2201 . If the weight is above the minimum level then in step 2208 the 
weight is gradually decreased for the path, then in step 2206 the weights for the 
remaining paths are adjusted accordingly to compensate (e.g., they are increased). 

If in step 2203 the quality of the path was greater than or equal to the 
threshold, then in step 2204 a check is made to determine whether the weight is less 

20 than a steady-state value for that path. If so, then in step 2205 the weight is increased 
toward the steady-state value, and irt step 2206 the weights for the remaining paths are 
adjusted accordingly to compensate (e.g., they are decreased). If in step 2204 the 
weight is not less than the steady-state value, then processing resumes at step 2201 
without adjusting the weights. 

25 The weights can be adjusted incrementally according to various functions, 

preferably by changing the value gradually. In one embodiment, a linearly decreasing 
function is used to adjust the weights; according to another embodiment, an 
exponential decay function is used. Gradually changing the weights helps to damp 
oscillators that might otherwise occur if the probabilities were abruptly. 
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Although not explicitly shown in FIG. 22A the process can be performed only 
periodically (e.g., according to a time schedule), or it can be continuously run, such as 
in a background mode of operation. In one embodiment, the combined weights of all 
potential paths should add up to unity (e.g., when the weighting for one path is 
decreased, the corresponding weights that the other paths will be selected will 
increase). 

Adjustments to weight values for other paths can be prorated. For example, a 
decrease of 10% in weight value for one path could result in an evenly distributed 
increase in the weights for the remaining paths. Alternatively, weightings could be 
adjusted according to a weighted formula as desired (e.g., favoring healthy paths over 
less healthy paths). In yet another variation, the difference in weight value can be 
amortized over the remaining links in a manner that is proportional to their traffic 
weighting. 

FIG. 22B shows steps that can be executed to shut down transmission links 
where a transmitter turns off. In step 2210, a transmitter shut-down event occurs. In 
step 221 1. a test is made to determine whether at least one transmitter is still turned 
on. If not, then in step 2215 all packets are dropped until a transmitter turns on. If in 
step 221 1 at least one transmitter is turned on, then in step 2212 the weight for the 
path is set to zero, and the weights for the remaining paths are adjusted accordingly. 

FIG. 23 shows a computer node 2301 employing various principles of the 
above-described embodiments. It is assumed that two computer nodes of the type 
shown in FIG. 23 communicate over a plurality of separate physical transmission 
paths. As shown in FIG. 23, four transmission paths XI through X4 are defined for 
communicating between the two nodes. Each node includes a packet transmitter 2302 
that operates in accordance with a transmit table 2308 as described above. (The 
packet transmitter could also operate without using the IP-hopping features described 
above, but the following description assumes that some form of hopping is employed 
in conjunction with the path selection mechanism.). The computer node also includes 
a packet receiver 2303 that operates in accordance with a receive table 2309, 
including a moving window W that moves as valid packets are received. Invalid 
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packets having source and destination addresses that do not fall within window W are 
rejected. 

As each packet is readied for transmission, source and destination IP addresses 
(or other discriminator values) are selected from transmit table 2308 according to any 
of the various algorithms described above, and packets containing these 
source/destination address pairs, which correspond to the node to which the four 
transmission paths are linked, are generated to a transmission path switch 2307. 
Switch 2307, which can comprise a software function, selects from one of the 
available transmission paths according to a weight distribution table 2306. For 
example, if the weight for path XI is 0.2, then every fifth packet will be transmitted 
on path XI. A similar regime holds true for the other paths as shown. Initially, each 
link's weight value can be set such that it is proportional to its bandwidth, which will 
be referred to as its "steady-state" value. 

Packet receiver 2303 generates an output to a link quality measurement 
function 2304 that operates as described above to determine the quality of each 
transmission path. (The input to packet receiver 2303 for receiving incoming packets 
is omitted for clarity). Link quality measurement function 2304 compares the link 
quality to a threshold for each transmission link and, if necessary, generates an output 
to weight adjustment function 2305. If a weight adjustment is required, then the 
weights in table 2306 are adjusted accordingly, preferably according to a gradual 
(e.g., linearly or exponentially declining) function. In one embodiment, the weight 
values for all available paths are initially set to the same value, and only when paths 
degrade in quality are the weights changed to reflect differences. 

Link quality measurement function 2304 can be made to operate as part of a 
synchronizer function as described above. That is, if resynchronization occurs and the 
receiver detects that synchronization has been lost (e.g., resulting in the 
synchronization window W being advanced out of sequence), that fact can be used to 
drive link quality measurement function 2304. According to one embodiment, load 
balancing is performed using information garnered during the normal 
synchronization, augmented slightly to communicate link health from the receiver to 
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the transmitter. The receiver maintains a count, MESS_R(W), of the messages 
received in synchronization window W. When it receives a synchronization request 
(SYNC_REQ) corresponding to the end of window W, the receiver includes counter 
MESSR in the resulting synchronization acknowledgement (SYNCACK) sent back 
5 to the transmitter. This allows the transmitter to compare messages sent to messages 
received in order to asses the health of the link. 

If synchronization is completely lost, weight adjustment function 2305 
decreases the weight value on the affected path to zero. When synchronization is 
regained, the weight value for the affected path is gradually increased to its original 
10 value. Alternatively, link quality can be measured by evaluating the length of time 
required for the receiver to acknowledge a synchronization request. In one 
embodiment, separate transmit and receive tables are used for each transmission path. 

When the transmitter receives a SYNC ACK, the MESS R is compared with 
the number of messages transmitted in a window (MESS T). When the transmitter 
15 receives a SYNC ACK, the traffic probabilities will be examined and adjusted if 
necessary. MESS_R is compared with the number of messages transmitted in a 
window (MESS_T). There are two possibilities: 

1. If MESS_R is less than a threshold value, THRESH, then the link will be 
deemed to be unhealthy. If the transmitter was turned off, the transmitter is turned on 

20 and the weight P for that link will be set to a minimum value MIN. This will keep a 
trickle of traffic on the link for monitoring purposes until it recovers. If the transmitter 
was turned on, the weight P for that link will be set to: 

P*=axMIN +(l-a)xP(l) 
Equation 1 will exponentially damp the traffic weight value to MIN during sustained 

25 periods of degraded service. 

2. If MESS_R for a link is greater than or equal to THRESH, the link will be 
deemed healthy. If the weight P for that link is greater than or equal to the steady 
state value S for that link, then P is left unaltered. If the weight P for that link is less 
than THRESH then P will be set to: 

30 P ? =(3x S+(l-p)xP (2) 
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where p is a parameter such that 0<=p<=l that determines the damping rate of P. 

Equation 2 will increase the traffic weight to S during sustained periods of 
acceptable service in a damped exponential fashion. 

A detailed example will now be provided with reference to FIG. 24. As 
5 shown in FIG. 24, a first computer 2401 communicates with a second computer 2402 
through two routers 2403 and 2404. Each router is coupled to the other router through 
three transmission links. As described above, these may be physically diverse links or 
logical links (including virtual private networks). 

Suppose that a first link LI can sustain a transmission bandwidth of 100 Mb/s 
10 and has a window size of 32; link L2 can sustain 75 Mb/s and has a window size of 
24; and link L3 can sustain 25 Mb/s and has a window size of 8. The combined links 
can thus sustain 200Mb/s. The steady state traffic weights are 0.5 for link LI; 0.375 
for link L2. and 0.125 for link L3. MIN=lMb/s, THRESH =0.8 MESS_T for each 
link, ot= 75 and p=.5. These traffic weights will remain stable until a link stops for 
15 synchronization or reports a number of packets received less than its THRESH. 
Consider the following sequence of events: 

1. Link LI receives a SYNC_ACK containing a MESS_R of 24, indicating 
that only 75% of the MESST (32) messages transmitted in the last window were 
successfully received. Link 1 would be below THRESH (0.8). Consequently, link 

20 LTs traffic weight value would be reduced to 0.12825. while link L2's traffic weight 
value would be increased to 0.65812 and link L3 ? s traffic weight value would be 
increased to 0.217938. 

2. Link L2 and L3 remained healthy and link LI stopped to synchronize. Then 
link LI 's traffic weight value would be set to 0, link L2's traffic weight value would 

25 be set to 0.75, and link L33*s traffic weight value would be set to 0.25. 

3. Link LI finally received a SYNC_ACK containing a MESS_R of 0 
indicating that none of the MESS_T (32) messages transmitted in the last window 
were successfully received. Link LI would be below THRESH. Link LTs traffic 
weight value would be increased to .005, link L2's traffic weight value would be 
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decreased to 0.74625, and link L3 ? s traffic weight value would be decreased to 
0.24875. 

4. Link LI received a SYNC_ACK containing a MESS_R of 32 indicating 
that 100% of the MESS_T (32) messages transmitted in the last window were 

5 successfully received. Link LI would be above THRESH. Link LPs traffic weight 
value would be increased to 0.2525, while link L2's traffic weight value would be 
decreased to 0.560625 and link L3's traffic weight value would be decreased to 
.186875. 

5. Link LI received a SYNCACK containing a MESS_R of 32 indicating 
10 that 100% of the MESS_T (32) messages transmitted in the last window were 

successfully received. Link LI would be above THRESH. Link Li's traffic weight 
value would be increased to 0.37625; link L2*s traffic weight value would be 
decreased to 0.4678125, and link L3's traffic weight value would be decreased to 
0.155937.5. 

15 6. Link LI remains healthy and the traffic probabilities approach their steady 

state traffic probabilities. 

B. Use of a DNS Proxy to Transparently Create Virtual Private Networks 

A second improvement concerns the automatic creation of a virtual private 
network (VPN) in response to a domain-name server look-up function. 

20 Conventional Domain Name Servers (DNSs) provide a look-up function that 

returns the IP address of a requested computer or host. For example, when a 
computer user types in the web name "Yahoo.com/ 5 the usefs web browser transmits 
a request to a DNS, which converts the name into a four-part IP address that is 
returned to the user's browser and then used by the browser to contact the destination 

25 web site. 

This conventional scheme is shown in FIG. 25. A user's computer 2501 
includes a client application 2504 (for example, a web browser) and an IP protocol 
stack 2505. When the user enters the name of a destination host, a request DNS REQ 
is made (through IP protocol stack 2505) to a DNS 2502 to look up the IP address 
30 associated with the name. The DNS returns the IP address DNS RESP to client 
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application 2504, which is then able to use the IP address to communicate with the 
host 2503 through separate transactions such as PAGE REQ and PAGE RESP. 

In the conventional architecture shown in FIG. 25, nefarious listeners on the 
Internet could intercept the DNS REQ and DNS RESP packets and thus learn what IP 
5 addresses the user was contacting. For example, if a user wanted to set up a secure 
communication path with a web site having the name "Target.com," when the user's 
browser contacted a DNS to find the IP address for that web site, the true IP address 
of that web site would be revealed over the Internet as part of the DNS inquiry. This 
would hamper anonymous communications on the Internet. 

10 One conventional scheme that provides secure virtual private networks over 

the Internet provides the DNS server with the public keys of the machines that the 
DNS server has the addresses for. This allows hosts to retrieve automatically the 
public keys of a host that the host is to communicate with so that the host can set up a 
VPN without having the user enter the public key of the destination host. One 

15 implementation of this standard is presently being developed as part of the 
FreeS/WAN project(RFC 2535). 

The conventional scheme suffers from certain drawbacks. For example, any 
user can perform a DNS request. Moreover, DNS requests resolve to the same value 
for all users. 

20 According to certain aspects of the invention, a specialized DNS server traps 

DNS requests and, if the request is from a special type of user (e.g., one for which 
secure communication services are defined), the server does not return the true IP 
address of the target node, but instead automatically sets up a virtual private network 
between the target node and the user. The VPN is preferably implemented using the 

25 IP address "hopping" features of the basic invention described above, such that the 
true identity of the two nodes cannot be determined even if packets during the 
communication are intercepted. For DNS requests that are determined to not require 
secure services (e.g., an unregistered user), the DNS server transparently "passes 
through" the request to provide a normal look-up function and return the IP address of 

30 the target web server, provided that the requesting host has permissions to resolve 

60 



WO 01/61922 



r^i/uaui/ir*tjHu 



unsecured sites. Different users who make an identical DNS request could be 
provided with different results. 

FIG. 26 shows a system employing various principles summarized above. A 
user's computer 2601 includes a conventional client (e.g., a web browser) 2605 and 
5 an IP protocol stack 2606 that preferably operates in accordance with an IP hopping 
function 2607 as outlined above. A modified DNS server 2602 includes a 
conventional DNS server function 2609 and a DNS proxy 2610. A gatekeeper server 
2603 is interposed between the modified DNS server and a secure target site 2704. 
An ''unsecure" target site 261 1 is also accessible via conventional IP protocols. 

10 According to one embodiment, DNS proxy 2610 intercepts all DNS lookup 

functions from client 2605 and determines whether access to a secure site has been 
requested. If access to a secure site has been requested (as determined, for example, 
by a domain name extension, or by reference to an internal table of such sites), DNS 
proxy 2610 determines whether the user has sufficient security privileges to access the 

15 site. If so. DNS proxy 2610 transmits a message to gatekeeper 2603 requesting that a 
virtual private network be created between user computer 2601 and secure target site 
2604. In one embodiment, gatekeeper 2603 creates ^hopblocks" to be used by 
computer 2601 and secure target site 2604 for secure communication. Then, 
gatekeeper 2603 communicates these to user computer 2601 . Thereafter, DNS proxy 

20 2610 returns to user computer 2601 the resolved address passed to it by the 
gatekeeper (this address could be different from the actual target computer) 2604, 
preferably using a secure administrative VPN. The address that is returned need not 
be the actual address of the destination computer. 

Had the user requested lookup of a non-secure web site such as site 2611, 

25 DNS proxy would merely pass through to conventional DNS server 2609 the look-up 
request, which would be handled in a conventional manner, returning the IP address 
of non-secure web site 261 1 . If the user had requested lookup of a secure web site but 
lacked credentials to create such a connection, DNS proxy 2610 would return a ''host 
unknown" error to the user. In this manner, different users requesting access to the 

30 same DNS name could be provided with different look-up results. 
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Gatekeeper 2603 can be implemented on a separate computer (as shown in 
FIG. 26) or as a function within modified DNS server 2602. In general, it is 
anticipated that gatekeeper 2703 facilitates the allocation and exchange of information 
needed to communicate securely, such as using "hopped" IP addresses. Secure hosts 
5 such as site 2604 are assumed to be equipped with a secure communication function 
such as an IP hopping function 2608. 

It will be appreciated that the functions of DNS proxy 2610 and DNS server 
2609 can be combined into a single server for convenience. Moreover, although 
element 2602 is shown as combining the functions of two servers, the two servers can 

10 be made to operate independently. 

FIG. 27 shows steps that can be executed by DNS proxy server 2610 to handle 
requests for DNS look-up for secure hosts. In step 2701, a DNS look-up request is 
received for a target host. In step 2702, a check is made to determine whether access 
to a secure host was requested. If not, then in step 2703 the DNS request is passed to 

15 conventional DNS server 2609, which looks up the IP address of the target site and 
returns it to the user's application for further processing. 

In step 2702, if access to a secure host was requested, then in step 2704 a 
further check is made to determine whether the user is authorized to connect to the 
secure host. Such a check can be made with reference to an internally stored list of 

20 authorized IP addresses, or can be made by communicating with gatekeeper 2603 
(e.g., over an "administrative" VPN that is secure). It will be appreciated that 
different levels of security can also be provided for different categories of hosts. For 
example, some sites may be designated as having a certain security level, and the 
security level of the user requesting access must match that security level. The user's 

25 security level can also be determined by transmitting a request message back to the 
user's computer requiring that it prove that it has sufficient privileges. 

If the user is not authorized to access the secure site, then a "host unknown" 
message is returned (step 2705). If the user has sufficient security privileges, then in 
step 2706 a secure VPN is established between the user's computer and the secure 

30 target site. As described above, this is preferably done by allocating a hopping regime 



that will be carried out between the user's computer and the secure target site, and is 
preferably performed transparently to the user (i.e., the user need not be involved in 
creating the secure link). As described in various embodiments of this application, 
any of various fields can be "hopped" (e.g., IP source/destination addresses; a field in 
5 the header; etc.) in order to communicate securely. 

Some or all of the security functions can be embedded in gatekeeper 2603, 
such that it handles all requests to connect to secure sites. In this embodiment, DNS 
proxy 2610 communicates with gatekeeper 2603 to determine (preferably over a 
secure administrative VPN) whether the user has access to a particular web site. 
10 Various scenarios for implementing these features are described by way of 

example below: 

Scenario #1 : Client has permission to access target computer, and gatekeeper 
has a rule to make a VPN for the client. In this scenario, the client's DNS request 
would be received by the DNS proxy server 261 0, which would forward the request to 

15 gatekeeper 2603. The gatekeeper would establish a VPN between the client and the 
requested target. The gatekeeper would provide the address of the destination to the 
DNS proxy, which would then return the resolved name as a result. The resolved 
address can be transmitted back to the client in a secure administrative VPN. 

Scenario #2 : Client does not have permission to access target computer. In 

20 this scenario, the client's DNS request would be received by the DNS proxy server 
2610, which would forward the request to gatekeeper 2603. The gatekeeper would 
reject the request, informing DNS proxy server 2610 that it was unable to find the 
target computer. The DNS proxy 2610 would then return a "host unknown" error 
message to the client. 

25 Scenario #3 : Client has permission to connect using a normal non-VPN link, 

and the gatekeeper does not have a rule to set up a VPN for the client to the target site. 
In this scenario, the client's DNS request is received by DNS proxy server 2610, 
which would check its rules and determine that no VPN is needed. Gatekeeper 2603 
would then inform the DNS proxy server to forward the request to conventional DNS 
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server 2609, which would re solve the request and return the result to the DNS proxy 
server and then back to the client. 

Scenario #4 : Client docs not have permission to establish a normal/non-VPN 
link, and the gatekeeper does not have a rule to make a VPN for the client to the target 
5 site. In this scenario, the DNS proxy server would receive the client's DNS request 
and forward it to gatekeeper 2603. Gatekeeper 2603 would determine that no special 
VPN was needed, but that the client is not authorized to communicate with non-VPN 
members. The gatekeeper would reject the request, causing DNS proxy server 2610 
to return an error message to the client. 

10 C. Large Link to Small Link Bandwidth Management 

One feature of the basic architecture is the ability to prevent so-called "denial 
of service" attacks that can occur if a computer hacker floods a known Internet node 
with packets, thus preventing the node from communicating with other nodes. 
Because IP addresses or other fields are "hopped" and packets arriving with invalid 

15 addresses are quickly discarded, Internet nodes are protected against flooding targeted 
at a single IP address. 

In a system in which a computer is coupled through a link having a limited 
bandwidth (e.g., an edge router) to a node that can support a much higher-bandwidth 
link (e.g., an Internet Service Provider), a potential weakness could be exploited by a 

20 determined hacker. Referring to FIG. 28, suppose that a first host computer 2801 is 
communicating with a second host computer 2804 using the IP address hopping 
principles described above. The first host computer is coupled through an edge router 
2802 to an Internet Service Provider (ISP) 2803 through a low bandwidth link (LOW 
BW), and is in turn coupled to second host computer 2804 through parts of the 

25 Internet through a high bandwidth link (HIGH BW). In this architecture, the ISP is 
able to support a high bandwidth to the internet, but a much lower bandwidth to the 
edge router 2802. 

Suppose that a computer hacker is able to transmit a large quantity of dummy 
packets addressed to first host computer 2801 across high bandwidth link HIGH BW. 
30 Normally, host computer 2801 would be able to quickly reject the packets since they 



would not fall within the acceptance window permitted by the IP address hopping 
scheme. However, because the packets must travel across low bandwidth link LOW 
BW, the packets overwhelm the lower bandwidth link before they are received by 
host computer 2801. Consequently, the link to host computer 2801 is effectively 
5 flooded before the packets can be discarded. 

According to one inventive improvement, a "link guard" function 2805 is 
inserted into the high-bandwidth node (e.g.. ISP 2803) that quickly discards packets 
destined for a low-bandwidth target node if they are not valid packets. Each packet 
destined for a low-bandwidth node is cryptographically authenticated to determine 

10 whether it belongs to a VPN. If it is not a valid VPN packet, the packet is discarded 
at the high-bandwidth node. If the packet is authenticated as belonging to a VPN, the 
packet is passed with high preference. If the packet is a valid non- VPN packet, it is 
passed with a lower quality of service (e.g.. lower priority). 

In one embodiment, the ISP distinguishes between VPN and non-VPN packets 

15 using the protocol of the packet. In the case of IPSEG [rfc 2401 ]. the packets have IP 
protocols 420 and 421. In the case of the TARP VPN. the packets will have an IP 
protocol that is not yet defined. The ISP's link guard, 2805, maintains a table of valid 
VPNs which it uses to validate whether VPN packets are cryptographically valid. 
According to one embodiment, packets that do not fall within any hop 

20 windows used by nodes on the low-bandwidth link are rejected, or are sent with a 
lower quality of service. One approach for doing this is to provide a copy of the IP 
hopping tables used by the low-bandwidth nodes to the high-bandwidth node, such 
that both the high-bandwidth and low-bandwidth nodes track hopped packets (e.g., the 
high-bandwidth node moves its hopping window as valid packets are received). In 

25 such a scenario, the high-bandwidth node discards packets that do not fall within the 
hopping window before they are transmitted over the low-bandwidth link. Thus, for 
example. ISP 2903 maintains a copy 2910 of the receive table used by host computer 
2901. Incoming packets that do not fail within this receive table are discarded. 
According to a different embodiment, link guard 2805 validates each VPN packet 

30 using a keyed hashed message authentication code (HMAC) [rfc 21 04]. According 
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to another embodiment, separate VPNs (using, for example, hopblocks) can be 
established for communicating between the low-bandwidth node and the high- 
bandwidth node (i.e., packets arriving at the high-bandwidth node are converted into 
different packets before being transmitted to the low-bandwidth node). 
5 As shown in FIG. 29, for example, suppose that a first host computer 2900 is 

communicating with a second host computer 2902 over the Internet, and the path 
includes a high bandwidth link HIGH BW to an ISP 2901 and a low bandwidth link 
LOW BW through an edge router 2904. In accordance with the basic architecture 
described above, first host computer 2900 and second host computer 2902 would 

10 exchange hopblocks (or a hopblock algorithm) and would be able to create matching 
transmit and receive tables 2905, 2906, 2912 and 2913. Then in accordance with the 
basic architecture, the two computers would transmit packets having seemingly 
random IP source and destination addresses, and each would move a corresponding 
hopping window in its receive table as valid packets were received. 

15 Suppose that a nefarious computer hacker 2903 was able to deduce that 

packets having a certain range of IP addresses (e.g., addresses 100 to 200 for the sake 
of simplicity) are being transmitted to ISP 2901, and that these packets are being 
forwarded over a low-bandwidth link. Hacker computer 2903 could thus "flood" 
packets having addresses failing into the range 100 to 200, expecting that they would 

20 be forwarded along low bandwidth link LOW BW. thus causing the low bandwidth 
link to become overwhelmed. The fast packet reject mechanism in first host computer 
3000 would be of little use in rejecting these packets, since the low bandwidth link 
was effectively jammed before the packets could be rejected. In accordance with one 
aspect of the improvement, however, VPN link guard 291 1 would prevent the attack 

25 from impacting the performance of VPN traffic because the packets would either be 
rejected as invalid VPN packets or given a lower quality of service than VPN traffic 
over the lower bandwidth link. A denial-of-service flood attack could, however, still 
disrupt non-VPN traffic. 

According to one embodiment of the improvement, ISP 2901 maintains a 

30 separate VPN with first host computer 2900, and thus translates packets arriving at the 
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ISP into packets having a different IP header before they are transmitted to host 
computer 2900. The cryptographic keys used to authenticate VPN packets at the link 
guard 291 1 and the cryptographic keys used to encrypt and decrypt the VPN packets 
at host 2902 and host 2901 can be different, so that link guard 2911 does not have 
5 access to the private host data; it only has the capability to authenticate those packets. 

According to yet a third embodiment, the low-bandwidth node can transmit a 
special message to the high-bandwidth node instructing it to shut down all 
transmissions on a particular IP address, such that only hopped packets will pass 
through to the low-bandwidth node. This embodiment would prevent a hacker from 

10 flooding packets using a single IP address. According to yet a fourth embodiment, the 
high-bandwidth node can be configured to discard packets transmitted to the low- 
bandwidth node if the transmission rate exceeds a certain predetermined threshold for 
any given IP address; this would allow hopped packets to go through. In this respect, 
link guard 291 1 can be used to detect that the rate of packets on a given IP address are 

15 exceeding a threshold rate; further packets addressed to that same IP address would be 
dropped or transmitted at a lower priority (e.g., delayed). 

D. Traffic Limiter 
In a system in which multiple nodes are communicating using "hopping^ 
technology, a treasonous insider could internally flood the system with packets. In 

20 order to prevent this possibility, one inventive improvement involves setting up 
"contracts" between nodes in the system, such that a receiver can impose a bandwidth 
limitation on each packet sender. One technique for doing this is to delay acceptance 
of a checkpoint synchronization request from a sender until a certain time period (e.g., 
one minute) has elapsed. Each receiver can effectively control the rate at which its 

25 hopping window moves by delaying "SYNC ACIC responses to "SYNC_REQ" 
messages. 

A simple modification to the checkpoint synchronizer will serve to protect a 
receiver from accidental or deliberate overload from an internally treasonous client. 
This modification is based on the observation that a receiver will not update its tables 
30 until a SYNC_REQ is received on hopped address CKPT_N. It is a simple matter of 
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deferring the generation of a new CKPT_N until an appropriate interval after previous 
checkpoints. 

Suppose a receiver wished to restrict reception from a transmitter to 100 
packets a second, and that checkpoint synchronization messages were triggered every 
5 50 packets. A compliant transmitter would not issue new S YNCREQ messages more 
often than every 0.5 seconds. The receiver could delay a non-compliant transmitter 
from synchronizing by delaying the issuance of CKPTN for 0.5 second after the last 
SYNCREQ was accepted. 

In general, if M receivers need to restrict N transmitters issuing new 

10 SYNCREQ messages after every W messages to sending R messages a second in 
aggregate, each receiver could defer issuing a new CKPT N until MxNxW/R seconds 
have elapsed since the last SYNC REQ has been received and accepted. If the 
transmitter exceeds this rate between a pair of checkpoints, it will issue the new 
checkpoint before the receiver is ready to receive it, and the SYNC_REQ will be 

15 discarded by the receiver. After this, the transmitter will re-issue the SYNC REQ 
every Tl seconds until it receives a SYNCACK. The receiver will eventually update 
CKPT N and the SYNC REQ will be acknowledged. If the transmission rate greatly 
exceeds the allowed rate, the transmitter will stop until it is compliant. If the 
transmitter exceeds the allowed rate by a little, it will eventually stop after several 

20 rounds of delayed synchronization until it is in compliance. Hacking the transmitter's 
code to not shut off only permits the transmitter to lose the acceptance window. In 
this case it can recover the window and proceed only after it is compliant again. 

Two practical issues should be considered when implementing the above 
scheme: 

25 1. The receiver rate should be slightly higher than the permitted rate in order 

to allow for statistical fluctuations in traffic arrival times and non-uniform load 
balancing. 

2. Since a transmitter will rightfully continue to transmit for a period after a 
SYNC_REQ is transmitted, the algorithm above can artificially reduce the 
30 transmitter's bandwidth. If events prevent a compliant transmitter from synchronizing 
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for a period (e.g. the network dropping a SYNCJIEQ or a SYNC_ACK) a 
SYNC_REQ will be accepted later than expected. After this, the transmitter will 
transmit fewer than expected messages before encountering the next checkpoint. The 
new checkpoint will not have been activated and the transmitter will have to 
5 retransmit the SYNC_REQ. This will appear to the receiver as if the transmitter is not 
compliant. Therefore, the next checkpoint will be accepted late from the transmitter's 
perspective. This has the effect of reducing the transmitters allowed packet rate until 
the transmitter transmits at a packet rate below the agreed upon rate for a period of 
time. 

10 To guard against this, the receiver should keep track of the times that the last 

C SYNCREQs were received and accepted and use the minimum of MxNxW/R 
seconds after the last SYNC_REQ has been received and accepted, 2xMxNxW/R 
seconds after next to . the last SYNC_REQ has been received and accepted, 
CxMxNxW/R seconds after (C-l) th to the last SYNC_REQ has been received, as the 

15 time to activate CKPTN. This prevents the receiver from inappropriately limiting 
the transmitters packet rate if at least one out of the last C SYNC REQs was 
processed on the first attempt. 

FIG. 30 shows a system employing the above-described principles. In FIG. 
30, two computers 3000 and 3001 are assumed to be communicating over a network 

20 N in accordance with the "hopping" principles described above (e.g., hopped IP 
addresses, discriminator values, etc.). For the sake of simplicity, computer 3000 will 
be referred to as the receiving computer and computer 3001 will be referred to as the 
transmitting computer, although full duplex operation is of course contemplated. 
. Moreover, although only a single transmitter is shown, multiple transmitters can 

25 transmit to receiver 3000. 

As described above, receiving computer 3000 maintains a receive table 3002 
including a window W that defines valid IP address pairs that will be accepted when 
appearing in incoming data packets. Transmitting computer 3001 maintains a 
transmit table 3003 from which the next IP address pairs will be selected when 

30 transmitting a packet to receiving computer 3000. (For the sake of illustration, 
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window W is also illustrated with reference to transmit table 3003). As transmitting 
computer moves through its table, it will eventually generate a SYNC_REQ message 
as illustrated in function 3010. This is a request to receiver 3000 to synchronize the 
receive table 3002, from which transmitter 3001 expects a response in the form of a 
5 CKPTN (included as part of a SYNC^ACK message). If transmitting computer 
3001 transmits more messages than its allotment, it will prematurely generate the 
SYNC_REQ message. (If it has been altered to remove the SYNC_REQ message 
generation altogether, it will fall out of synchronization since receiver 3000 will 
quickly reject packets that fall outside of window W, and the extra packets generated 

10 by transmitter 3001 will be discarded). 

In accordance with the improvements described above, receiving computer 
3000 performs certain steps when a SYNCREQ message is received, as illustrated in 
FIG. 30. In step 3004, receiving computer 3000 receives the SYNC_REQ message. 
In step 3005, a check is made to determine whether the request is a duplicate. If so, it 

15 is discarded in step 3006. In step 3007, a check is made to determine whether the 
SYNCREQ received from transmitter 3001 was received at a rate that exceeds the 
allowable rate R (i.e., the period between the time of the last SYNC REQ message). 
The value R can be a constant, or it can be made to fluctuate as desired. If the rate 
exceeds R. then in step 3008 the next activation of the next CKPTN hopping table 

20 entry is delayed by W/R seconds after the last SYNCREQ has been accepted. 

Otherwise, if the rate has not been exceeded, then in step 3109 the next 
CKPTN value is calculated and inserted into the receiver's hopping table prior to the 
next SYNC REQ from thetransmitter 3101. Transmitter 3101 then processes the 
SYNC_REQ in the normal manner. 

25 E. Signaling Synchronizer 

In a system in which a large number of users communicate with a central node 
using secure hopping technology, a large amount of memory must be set aside for 
hopping tables and their supporting data structures. For example, if one million 
subscribers to a web site occasionally communicate with the web site, the site must 

30 maintain one million hopping tables, thus using up valuable computer resources, even 
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though only a small percentage of the users may actually be using the system at any 
one time. A desirable solution would be a system that permits a certain maximum 
number of simultaneous links to be maintained, but which would "recognize" millions 
of registered users at any one time. In other words, out of a population of a million 
5 registered users, a few thousand at a time could simultaneously communicate with a 
central server, without requiring that the server maintain one million hopping tables of 
appreciable size. 

One solution is to partition the central node into two nodes: a signaling server 
that performs session initiation for user log-on and log-off (and requires only 

10 minimally sized tables), and a transport server that contains larger hopping tables for 
the users. The signaling server listens for the millions of known users and performs a 
fast-packet reject of other (bogus) packets. When a packet is received from a known 
user, the signaling server activates a virtual private link (VPL) between the user and 
the transport server, where hopping tables are allocated and maintained. When the 

15 user logs onto the signaling server, the user's computer is provided with hop tables for 
communicating with the transport server, thus activating the VPL. The VPLs can be 
torn down when they become inactive for a time period, or they can be torn down 
upon user log-out. Communication with the signaling server to allow user log-on and 
log-off can be accomplished using a specialized version of the checkpoint scheme 

20 described above. 

FIG. 31 shows a system employing certain of the above-described principles. 
In FIG. 31. a signaling server 3101 and a transport server 3102 communicate over a 
link. Signaling server 3101 contains a large number of small tables 3106 and 3107 
that contain enough information to authenticate a communication request with one or 

25 more clients 3103 and 3104. As described in more detail below, these small tables 
may advantageously be constructed as a special case of the synchronizing checkpoint 
tables described previously. Transport server 3102, which is preferably a separate 
computer in communication with signaling server 3101, contains a smaller number of 
larger hopping tables 3108,31 09, and 3110 that can be allocated to create a VPN with 

30 one of the client computers. 
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According to one embodiment, a client that has previously registered with the 
system (e.g., via a system administration function, a user registration procedure, or 
some other method) transmits a request for information from a computer (e.g., a web 
site). In one variation, the request is made using a "hopped" packet, such that 
5 signaling server 3101 will quickly reject invalid packets from unauthorized computers 
such as hacker computer 3105. An "administrative" VPN can be established between 
all of the clients and the signaling server in order to ensure that a hacker cannot flood 
signaling server 31 01 with bogus packets. Details of this scheme are provided below. 
Signaling server 3101 receives the request 3111 and uses it to determine that 

10 client 3 1 03 is a validly registered user. Next, signaling server 3101 issues a request to 
transport server 3102 to allocate a hopping table (or hopping algorithm or other 
regime) for the purpose of creating a VPN with client 3103. The allocated hopping 
parameters are returned to signaling server 3101 (path 3113), which then supplies the 
hopping parameters to client 3103 via path 3114, preferably in encrypted form. 

15 Thereafter, client 3103 communicates with transport server 3102 using the 

normal hopping techniques described above. It will be appreciated that although 
signaling server 3101 and transport server 3102 are illustrated as being two separate 
computers, they could of course be combined into a single computer and their 
functions performed on the single computer. Alternatively, it is possible to partition 

20 the functions shown in FIG. 31 differently from as shown without departing from the 
inventive principles. 

One advantage of the above-described architecture is that signaling server 
3101 need only maintain a small amount of information on a large number of 
potential users, yet it retains the capability of quickly rejecting packets from 

25 unauthorized users such as hacker computer 3105. Larger data tables needed to 
perform the hopping and synchronization functions are instead maintained in a 
transport server 3102, and a smaller number of these tables are needed since they are 
only allocated for "active" links. After a VPN has become inactive for a certain time 
period (e.g.. one hour), the VPN can be automatically torn down by transport server 

30 3 1 02 or signaling server 3101. 
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A more detailed description will now be provided regarding how a special 
case of the checkpoint synchronization feature can be used to implement the signaling 
scheme described above. 

The signaling synchronizer may be required to support many (millions) of 
5 standing, low bandwidth connections. It therefore should minimize per-VPL memory 
usage while providing the security offered by hopping technology. In order to reduce 
memory usage in the signaling server, the data hopping tables can be completely 
eliminated and data can be carried as part of the SYNCREQ message. The table used 
by the server side (receiver) and client side (transmitter) is shown schematically as 
10 element 3106 in FIG. 31. 

The meaning and behaviors of CKPTN, CKPT_0 and CKPTJl remain the 
same from the previous description, except that CKPTN can receive a combined data 
and SYNCJREQ message or a SYNC_REQ message without the data. 

The protocol is a straightforward extension of the earlier synchronizer. 
15 Assume that a client transmitter is on and the tables are synchronized. The initial 
tables can be generated **out of band/' For example, a client can log into a web server 
to establish an account over the Internet. The client will receive keys etc encrypted 
over the Internet. Meanwhile, the server will set up the signaling VPN on the 
signaling server. 

20 Assuming that a client application wishes to send a packet to the server on the 

client's standing signaling VPL: 

1 . The client sends the message marked as a data message on the inner header 
using the transmitter's CKPT_N address. It turns the transmitter off and starts a timer 
Tl noting CKPT_0. Messages can be one of three types: DATA, SYNC REQ and 

25 SYNC_ACK. In the normal algorithm, some potential problems can be prevented by 
identifying each message type as part of the encrypted inner header field. In this 
algorithm, it is important to distinguish a data packet and a SYNC_REQ in the 
signaling synchronizer since the data and the SYNCJIEQ come in on the same 
address. 
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2. When the server receives a data message on its CKPT_N, it verifies the 
message and passes it up the stack. The message can be verified by checking message 
type and and other information (i.e user credentials) contained in the inner header It 
replaces its CKPT_0 with CKPTN and generates the next CKPTN. It updates its 

5 transmitter side CKPT_R to correspond to the client's receiver side CKPT_R and 
transmits a S YNC_ACK containing CKPTO in its payload. 

3. When the client side receiver receives a SYNCACK on its CKPT_R with a 
payload matching its transmitter side CKPT O and the transmitter is off, the 
transmitter is turned on and the receiver side CKPT_R is updated. If the 

10 SYNCACK's payload does not match the transmitter side CKPT O or the 
transmitter is on, the SYNCACK is simply discarded. 

4. Tl expires: If the transmitter is off and the client's transmitter side 
CKPT_0 matches the CKPT O associated with the timer, it starts timer Tl noting 
CKPT O again, and a SYNC REQ is sent using the transmitter's CKPT O address. 

1 5 Otherwise, no action is taken. 

5. When the server receives a SYNC REQ on its CKPTJM, it replaces its 
CKPTJ) with CKPT_N and generates the next CKPT N. It updates its transmitter 
side CKPT_R to correspond to the client's receiver side CKPTR and transmits a 
SYNC_ACK containing CKPT_0 in its payload. 

20 6. When the server receives a SYNCJREQ on its CKPT O, it updates its 

transmitter side CKPT R to correspond to the client's receiver side CKPT_R and 
transmits a SYNC_ACK containing CKPT O in its payload. 

FIG. 32 shows message flows to highlight the protocol. Reading from top to 
bottom, the client sends data to the server using its transmitter side CKPT_N. The 

25 client side transmitter is turned off and a retry timer is turned off. The transmitter will 
not transmit messages as long as the transmitter is turned off. The client side 
transmitter then loads CKPTJM into CKPT_0 and updates CKPT_N. This message is 
successfully received and a passed up the stack. It also synchronizes the receiver i.e, 
the server loads CKPT_N into CKPT_0 and generates a new CKPTJM, it generates a 

30 new CKPT_R in the server side transmitter and transmits a SYNC^ACK containing 



the server side receiver's CKPTO the server. The SYNC_ACK is successfully 
received at the client. The client side receiver's CKPTR is updated, the transmitter is 
turned on and the retry timer is killed. The client side transmitter is ready to transmit a 
new data message. 

5 Next, the client sends data to the server using its transmitter side CKPTJSf. 

The client side transmitter is turned off and a retry timer is turned off. The transmitter 
will not transmit messages as long as the transmitter is turned off. The client side 
transmitter then loads CKPT_N into CKPT_0 and updates CKPT_N. This message is 
lost. The client side timer expires and as a result a SYNCREQ is transmitted on the 

10 client side transmitter's CKPT O (this will keep happening until the SYNC_ACK has 
been received at the client). The SYNC_REQ is successfully received at the server. It 
synchronizes the receiver i.e, the server loads CKPTN into CKPT O and generates a 
new CKPT N, it generates an new CKPT R in the server side transmitter and 
transmits a SYNC ACK containing the server side receiver's CKPTJ3 the server. 

15 The SYNC_ACK is successfully received at the client. The client side receiver's 
CKPT_R is updated, the transmitter is turned off and the retry timer is killed. The 
client side transmitter is ready to transmit a new data message. 

There are numerous other scenarios that follow this flow: For example, the 
SYNC_ACK could be lost. The transmitter would continue to re-send the 

20 S YNC REQ until the receiver synchronizes and responds. 

The above-described procedures allow a client to be authenticated at signaling 
server 3201 while maintaining the ability of signaling server 3201 to quickly reject 
invalid packets, such as might be generated by hacker computer 3205. In various 
embodiments, the signaling synchronizer is really a derivative of the synchronizer. It 

25 provides the same protection as the hopping protocol, and it does so for a large 
number of low bandwidth connections. 
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CLAIMS 

1. A method of transmitting data packets between a first computer and a 
second computer, wherein the first computer and the second computer are linked via a 
plurality of separate transmission paths, the method comprising the steps of: 
5 (1) assigning a weight value to each of the plurality of transmission paths, 

wherein each respective weight value represents the relative number of packets that a 
respective transmission path will transmit; 

(2) for each data packet that is to be transmitted from the first computer to the 
second computer, selecting one of the plurality of transmission paths on the basis of 

10 each respective transmission path's assigned weight value; 

(3) measuring the transmission quality for each of the plurality of transmission 
paths; and 

(4) adjusting downwardly to a non-zero value the assigned weight value for a 
transmission path for which the transmission quality has declined. 

15 2. The method of claim 1 , wherein step (4) comprises the step of gradually 

decreasing over time the assigned weight value in relation to weight values assigned 
to the remaining transmission paths. 

3. The method of claim 2, wherein step (4) comprises the step of gradually 
decreasing the assigned weight value according to an incrementally decreasing 

20 function. 

4. The method of claim 2, wherein step (4) comprises the step of gradually 
decreasing the assigned weight value according to an exponentially decaying 
function. 

5. The method of claim 1, wherein step (3) comprises the step of determining 
25 that one or more packets transmitted to the second computer was not acknowledged 

by the second computer. 

6. The method of claim 1 , wherein step (3) comprises the step of evaluating 
the contents of a synchronization packet that maintains synchronization with a 
moving window of valid values. 
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7. The method of claim 1, further comprising the step of inserting into each 
data packet a source and destination IP address pair that is selected according to a 
pseudo-random sequence. 

8. The method of claim 1 . wherein step (4) comprises the step of adjusting 

5 downwardly the assigned weight value for a transmission path only if the transmission 
quality has declined below a predetermined threshold. 

9. The method of claim 1, further comprising the step of adjusting upwardly 
the assigned weight value that was adjusted in step (4) if it is later determined that the 
transmission quality has improved. 

10 10. The method of claim 1 , further comprising the step of adjusting upwardly 

the weight values of the remaining transmission links in an amount that compensates 
for the downwardly adjusted weight value. 

1 1 . The method of claim 1 0, wherein the step of adjusting upwardly 
comprises the step of equally distributing the amount that was downwardly adjusted 

1 5 across the remaining transmission links. 

12. The method of claim 1, further comprising the step of adjusting 
downwardly to zero the assigned weight value for any transmission link whose quality 
has degraded below a preset threshold. 

13. The method of claim 1. wherein steps (2) through (4) are repeated 
20 periodically. 

14. A first computer that transmits data packets to a second computer over a 
plurality of separate transmission paths, wherein the first computer performs the steps 
of: 

(1) assigning a weight value to each of the plurality of transmission paths, 

25 wherein each respective weight value represents the relative number of packets that a 
respective transmission path will transmit; 

(2) for each data packet that is to be transmitted to the second computer, 
selecting one of the plurality of transmission paths on the basis of each respective 
transmission path's assigned weight value; 
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(3) measuring the transmission quality for each of the plurality of transmission 
paths; and 

(4) adjusting downwardly to a non-zero value the assigned weight value for a 
transmission path for which the transmission quality has declined. 

5 15. The first computer of claim 14, wherein the first computer gradually 

decreases over time the assigned weight value in relation to weight values assigned to 
the remaining transmission paths. 

16. The first computer of claim 15, wherein the first computer gradually 
decreases the assigned weight value according to an incrementally decreasing 

10 function. 

1 7. The first computer of claim 1 5, wherein the first computer gradually 
decreases the assigned weight value according to an exponentially decaying function. 

1 8. The first computer of claim 1 4, wherein the first computer measures the 
transmission quality by determining that one or more packets transmitted to the 

1 5 second computer was not acknowledged by the second computer. 

19. The first computer of claim 14, wherein the first computer measures the 
transmission quality by evaluating the contents of a synchronization packet that 
maintains synchronization with a moving window of valid values. 

20. The first computer of claim 14, wherein the first computer inserts into 
20 each data packet a source and destination IP address pair that is selected according to 

a pseudo-random sequence. 

21 . The first computer of claim 14, wherein the first computer adjusts 
downwardly the assigned weight value for any transmission path only if the 
transmission quality has declined below a predetermined threshold. 

25 22. The first computer of claim 14, wherein the first computer adjusts 

upwardly the assigned weight value that was adjusted in step (4) if it is later 
determined that the transmission quality has improved. 

23. The first computer of claim 14, wherein the first computer adjusts 
upwardly the weight values of the remaining transmission links in an amount that 

30 compensates for the downwardly adjusted weight value. 
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24. The first computer of claim 23, wherein the first computer upwardly 
adjusts probabilities across the remaining transmission links in an amount equal to the 
downwardly adjusted weight value. 

25. The first computer of claim 14, wherein the first computer adjusts 

5 downwardly to zero the assigned weight value for any transmission link whose quality 
has degraded below a preset threshold. 

26. The first computer of claim 14, wherein the first computer repeats steps 
(2) through (4) periodically. 

27. A system comprising the first computer of claim 14 and a second 
10 computer constructed in accordance with the first computer of claim 14. 

28. A method of transparently creating a virtual private network (VPN) 
between a client computer and a target computer, comprising the steps of: 

(1) generating from the client computer a Domain Name Service (DNS) 
request that requests an IP address corresponding to a domain name associated with 

15 the target computer; 

(2) determining whether the DNS request transmitted in step (1) is requesting 
access to a secure web site; and 

(3) in response to determining that the DNS request in step (2) is requesting 
access to a secure target web site, automatically initiating the VPN between the client 

20 computer and the target computer. 

29. The method of claim 28, wherein steps (2) and (3) are performed at a 
DNS server separate from the client computer. 

30. The method of claim 28, further comprising the step of: 

(4) in response to determining that the DNS request in step (2) is not 

25 requesting access to a secure target web site, resolving the IP address for the domain 
name and returning the IP address to the client computer. 

31 . The method of claim 28, wherein step (3) comprises the step of, prior to 
automatically initiating the VPN between the client computer and the target computer, 
determining whether the client computer is authorized to establish a VPN with the 

30 target computer and, if not so authorized, returning an error from the DNS request. 
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32. The method of cl iim 28, wherein step (3) comprises the step of, prior to 
automatically initiating the VPN between the client computer and the target computer, 
determining whether the client computer is authorized to resolve addresses of non 
secure target computers and, if not so authorized, returning an error from the DNS 

5 request. 

33. The method of claim 28, wherein step (3) comprises the step of 
establishing the VPN by creating an IP address hopping scheme between the client 
computer and the target computer. 

34. The method of claim 28. wherein step (3) comprises the step of using a 
1 0 gatekeeper computer that allocates VPN resources for communicating between the 

client computer and the target computer. 

35. The method of claim 28, wherein step (2) is performed in a DNS proxy 
server that passes through the request to a DNS server if it is determined in step (3) 
that access is not being requested to a secure target web site. 

15 36. The method of claim 32, wherein step (3) comprises the step of 

transmitting a message to the client computer to determine whether the client 
computer is authorized to establish the VPN target computer. 

37. A system that transparently creates a virtual private network (VPN) 
between a client computer and a secure target computer, comprising: 

20 a DNS proxy server that receives a request from the client computer to look up 

an IP address for a domain name, wherein the DNS proxy server returns the IP 
address for the requested domain name if it is determined that access to a non-secure 
web site has been requested, and wherein the DNS proxy server generates a request to 
create the VPN between the client computer and the secure target computer if it is 

25 determined that access to a secure web site has been requested; and 

a gatekeeper computer that allocates resources for the VPN between the client 
computer and the secure web computer in response to the request by the DNS proxy 
server. 

38. The system of claim 37, wherein the gatekeeper computer creates the 
30 VPN by establishing an IP address hopping regime that is used to pseudorandomly 



change IP addresses in packets transmitted between the client computer and the secure 
target computer. 

39. The system of claim 37. wherein the gatekeeper computer determines 
whether the client computer has sufficient security privileges to create the VPN and, if 

5 the client computer lacks sufficient security privileges, rejecting the request to create 
the VPN. 

40. A method of preventing data packets received from a high bandwidth link 
from flooding a low bandwidth link, comprising the steps of: 

(1 ) receiving data packets from the high bandwidth link that are ostensibly 
10 addressed to a computer residing on the low-bandwidth link; 

(2) for each data packet, determining whether the data packet is validly 
addressed to the computer on the low-bandwidth link; 

(3) in response to determining that the data packet is not validly addressed to 
the computer on the low-bandwidth link, rejecting the data packet; and 

1 5 (4) in response to determining that the data packet is validly addressed to the 

computer on the low-bandwidth link, forwarding the data packet to the computer over 
the low-bandwidth link. 

41 . The method of claim 40, wherein step (3) comprises the step of comparing 
a value in a header of each data packet to a set of valid values maintained for the 

20 computer on the low-bandwidth link. 

42. The method of claim 41 , wherein step (3) comprises the step of comparing 
a value in a header of each data packet to a moving window of valid values. 

43. The method of claim 42. wherein step (3) comprises the step of comparing 
the IP address in the header of each data packet to a moving window of valid IP 

25 addresses, wherein the moving window is also maintained by the computer on the 
low-bandwidth link. 

44. The method of claim 40. wherein step (3) comprises the step of reducing a 
priority level of the packet in relation to other data packets, wherein the priority level 
determines whether a particular data packet will be transmitted before another data 

30 packet having a different priority level. 
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45. The method of claim 40, wherein step (3) comprises the step of 
performing a cryptographic check on each data packet to determine whether each data 
packet is validly addressed. 

46. The method of claim 40, wherein step (3) comprises the step of receiving 
5 a message from the computer on the low-bandwidth link to stop accepting messages 

having a particular characteristic. 

47. The method of claim 46, wherein step (3) comprises the step of receiving 
a message from the computer on the low-bandwidth link to stop accepting messages 
addressed to a particular IP address. 

1 0 48. The method of claim 40, wherein step (3) comprises the step of 

determining that a packet transmission rate has been exceeded for a given packet 
parameter. 

49. The method of claim 48, wherein step (3) comprises the step of 
determining that a packet transmission rate has been exceeded for a given IP 

15 destination address. 

50. In a system having a low bandwidth data link, a first computer coupled to 
the low bandwidth data link, and a high bandwidth data link, an improvement 
comprising: 

a second computer coupled between the low bandwidth data link and the high 
20 bandwidth data link, wherein the second computer receives data packets from the high 
bandwidth data link and, if they are addressed to the first computer, routes them to the 
first computer over the low bandwidth data link, 

wherein the second computer prevents invalid data packets ostensibly 
addressed to the first computer from being transmitted over the low bandwidth data 
25 link. 

51 . The system of claim 50, wherein the second computer prevents invalid 
data packets from being transmitted over the low bandwidth data link by comparing a 
discriminator field in a header of each data packet to a table of valid discriminator 
fields maintained for the first computer. 
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52. The system of claim 50, wherein the second computer compares an 
Internet Protocol (IP) address in a header of each data packet to a table of valid IP 
addresses. 

53. The system of claim 52, wherein the second computer compares the IP 
5 address in the header of each data packet to a moving window of valid IP addresses, 

wherein the moving window is also maintained by the first computer. 

54. The system of claim 50, wherein the second computer reduces a priority 
level of a data packet in relation to other data packets, wherein the priority level 
determines whether a particular data packet will be transmitted before another data 

10 packet having a different priority level. 

55. The system of claim 50, wherein the second computer performs a 
cryptographic check on each data packet to determine whether each data packet is 
validly addressed. 

56. The system of claim 50, wherein the second computer receives a message 
15 from the first computer that causes the second computer to stop accepting messages 

having a particular characteristic. 

57. The system of claim 56. wherein the second computer receiving a 
message from the first computer to stop accepting messages addressed to a particular 
IP address. 

20 58. The system of claim 50, wherein the second computer rejects invalid 

packets by determining that a packet transmission rate has been exceeded for a given 
packet parameter. 

59. The system of claim 58, wherein the second computer determines that a 
packet transmission rate has been exceeded for a given IP destination address. 

25 60. In a system comprising a first computer that transmits data packets to a 

second computer over a network according to a scheme by which at least one field in 
a series of data packets is periodically changed according to a sequence known by the 
first and second computers, and wherein the second computer periodically receives a 
synchronization request from the first computer to maintain synchronization of the 

30 sequence between the first and second computers, a method comprising the steps of: 
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(1) receiving at the fir:;t computer the synchronization request from the second 
computer; 

(2) determining whether the synchronization request was received in less than 
a predetermined interval; 

(3) in response to determining that the synchronization request was received in 
less than the predetermined interval, ignoring the synchronization request; and 

(4) in response to determining that the synchronization request was not 
received in less than the predetermined interval, providing the synchronization 
response to the first computer. 

61 . The method of claim 60, wherein step (3) comprises the step of delaying 
the acceptance of a SYNC_REQ for W/R seconds, where W is the number of data 
packets between synchronization requests according to an agreed schedule, and R is 
the agreed rate at which synchronization requests should be received according to the 
agreed schedule. 

62. The method of claim 60, further comprising the step of determining 
whether the synchronization request is a duplicate of a previously received 
synchronization request and, if it is a duplicate, discarding it. 

63. The method of claim 60, wherein step (4) comprises the step of providing 
a response that includes a new checkpoint for synchronizing a window in a hopping 
table. 

64. A computer that receives data packets from a second computer over a 
network according to a scheme by which at least one field in a series of data packets 
is periodically changed according to a known sequence, wherein the second computer 
periodically transmits a synchronization request to maintain synchronization of the 
sequence, wherein the computer performs the steps of: 

(1) receiving the synchronization request from the second computer; 

(2) determining whether the synchronization request was received in less than 
a predetermined interval; 

(3) in response to determining that the synchronization request was received in 
less than a predetermined interval ignoring the synchronization request; and 
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(4) in response to determining that the synchronization request was not 
received in less than a predetermined interval, providing the response to the first 
computer. 

65. The computer of claim 64, wherein the computer delays the acceptance of 
5 a S YNCREQ in step (3) for W/R seconds, where W is the number of data packets 

between synchronization requests according to an agreed schedule, and R is the 
agreed rate at which synchronization requests should be received according to the 
agreed schedule. 

66. The computer of claim 64, wherein the computer further performs the step 
10 of determining whether the synchronization request is a duplicate of a previously 

received synchronization request and, if it is a duplicate, discarding it. 

67. A method of establishing communication between one of a plurality of 
client computers and a central computer that maintains a plurality of authentication 
tables each corresponding to one of the client computers, the method comprising the 

15 steps of: 

( 1 ) in the central computer, receiving from one of the plurality of client 
computers a request to establish a connection; 

(2) authenticating, with reference to one of the plurality of authentication 
tables, that the request received in step (1) is from an authorized client; 

20 (3) responsive to a determination that the request is from an authorized client, 

allocating resources to establish a virtual private link between the client and a second 
computer; and 

(4) communicating between the authorized client and the second computer 
using the virtual private link. 
25 68. The method of claim 67, wherein step (4) comprises the step of 

communicating according to a scheme by which at least one field in a series of data 
packets is periodically changed according to a known sequence. 

69. The method of claim 68. wherein step (4) comprises the step of comparing 
an Internet Protocol (IP) address in a header of each data packet to a table of valid IP 
30 addresses maintained in a table in the second computer. 
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70. The method of claim 69, wherein step (4) comprises the step of comparing 
the IP address in the header of each data packet to a moving window of valid IP 
addresses, and rejecting data packets having IP addresses that do not fall within the 
moving window. 

5 71 . The method of claim 67, wherein step (2) comprises the step of using a 

checkpoint data structure that maintains synchronization of a periodically changing 
parameter known by the central computer and the client computer to authenticate the 
client. 
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